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(57)Abstract 

PROBLEM TO BE SOLVED: To accurately animate a feature of a 
face, and transform an expression of the face. 
SOLUTION: A code book stores a data defining a face expression 
set of a first person. A face training expression set from a second 
person and a corresponding expression from the code book are 
used to draw a transformation function, and the transformation 
function is applied for all the expressions in the code book. The 
expression from the first person is transformed into the expression 
of the second person with real feeling by this manner, and vice 
versa thereof. A single common generic face model is prepared to 
be used as a fundamental for an operation for adapting a large 
number of different faces. A mechanism allowing a corresponsence 
between the different faces is provided by using the face model and 
a kind of limitation defined by a user. This transformation processing 
is executed by a processing unit 132. 
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* NOTICES * 

JPO and NCI PI are not responsible for any 
damages caused by the use of this translation. 

1. This document has been translated by computer. So the translation may not reflect the original precisely. 

2. **** shows the word which can not be translated. 
3.1n the drawings, any words are not translated. 



[Claim(s)] 

[Claim 1] The code book which has stored the data which define the 1st face expression set of the 1st person is 
defined. The data which define the 2nd face expression set which gives a training expression set of the 2nd different 
person from the 1st person are prepared. Said training expression set. The face expression conversion approach 
characterized by drawing a transform function from the expression which corresponds from said 1 st expression set, 
and obtaining a synthetic expression set with the application of said transform function to this 1st expression set 
[Claim 2] It is the approach characterized by said training expression set containing an expression fewer than a 
code book in an approach according to claim 1. 

[Claim 3] It is the approach characterized by said transform function compensating the size of the face of the 1st 
person and the 2nd person, and the difference in a configuration in an approach according to claim 1. 
[Claim 4] The approach characterized by calculating the linearity-conversion to the expression set of another side 
from one expression set in an approach according to claim 1 at the step which draws said transform function. 
[Claim 5] In an approach according to claim 1, at the step which draws said transform function Each expression is 
expressed as a 3m vector containing x in the correlation sample location of m pieces, y, and z variation rates, n 
expression vector sets of the face changed are gal... The vector set to which it is given to as n and a target face 
corresponds is gb1... When given as n, they are 3m linearity least square system of the following format, and 
[Equation 1]. 



The approach characterized by calculating one linear prediction child set aj for every coordinate of ga by **** 
Lycium chinense. 

[Claim 6] The approach characterized by using only the subset of the point about each gaj in an approach according 
to claim 5 at said step to calculate. 

[Claim 7] The approach characterized by using only the correlation sample point set as the object of consideration, 
and the point which is sharing the edge in an approach according to claim 6 at said step to be used. 
[Claim 8] The approach characterized by controlling diffusion of a singular value when calculating a false inverse 
function further in an approach according to claim 5, in order to calculate the solution to. 

[Claim 9] The step which controls said diffusion in an approach according to claim 8 is an approach characterized by 
sigmal being the maximum singular value of a matrix in zero, nothing, and here about all less than alphasigmal 
singular values. 

[Claim 10] The approach characterized by what the 2nd persons face is irradiated with lighting and the structure 
data which have described the structure of a face, and the reflection factor data which have described the 
reflection factor property of the face from an exposure are simultaneously incorporated for at the step which 
prepares the data which define said 2nd face expression set in an approach according to claim 1. 
[Claim 1 1] In an approach according to claim 10 at said step to irradiate Use two or more light sources and one of 
the light sources of those projects a pattern on the 2nd person's face. At least one of nothing and said the light 
sources is the infrared light source as a check of the pattern to structure data is possible, and at least one of these 
the light sources polarizes. At said step to incorporate The approach characterized by using the camera equipped 
with the polarizer which inhibits specular light and incorporates diffusion component reflective data. 
[Claim 12] At the step which prepares the data which define the 2nd face expression set in an approach according 
to claim 1 The 2nd person's face is irradiated by the 1st polarization light source chosen so that the specular 
reflection suppression reflection factor property of a face could be checked. The approach characterized by 
irradiating the 2nd person's face by the 2nd structuring light source which projects a pattern on a face, and 
incorporating both specular reflection suppression reflective data and structure data from a simultaneous exposure 
to the exposure and coincidence while irradiating the face by the 1st polarization light source. 

[Claim 13] It is the approach characterized by outputting the light of the frequency from which said 1st polarization 
light source and said 2nd structuring light source differ in an approach according to claim 1 2. 

[Claim 14] It is the approach characterized by said 1st polarization light source and said 2nd structuring light source 
outputting infrared light in an approach according to claim 12. 

[Claim 15] In an approach according to claim 12, the incorporated data are processed further, and it is (a). The data 
which have described the dimension modality of a face, and (b) Approach characterized by obtaining both data which 
have described the diffuse reflection property of a face. 
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[Claim 16] At the step which prepares the data which define said 2nd face expression set in an approach according 
to claim 1 Irradiate the 2nd person's face by the light source from which plurality differs, and range map data are 
measured from said exposure. The approach characterized by measuring image data from this exposure, drawing a 
three-dimension side from said range map data, calculating the field normal to said three-dimension side, processing 
said field normal and said image data, and drawing an albedo map. 

[Claim 1 7] It is the approach characterized by at least one of two or more of the light sources polarizing in an 
approach according to claim 16. 

[Claim 18] It is the approach characterized by two or more light sources of all polarizing in an approach according to 
claim 16. 

[Claim 19] When it performs by computer, the training expression set from a certain person. The expression which 
corresponds from another person's code book is operated. Said training expression set The step which calculates 
the expression and spiral form-conversion corresponding to these, and the step which obtains a synthetic 
expression set with the application of a transform function from a code book to two or more expression, One or two 
media or more which are characterized by storing the instruction for performing a computer which can be computer 
read and which can be computer read. 

[Claim 20] It is the medium which is characterized by for said instruction which can be computer read making a 
synthetic expression set use it for a computer in the medium according to claim 19 which can be computer read, 
and transforming the expression from one person to the expression of the person of another side and which can be 
computer read. 

[Claim 21] The expression which said instruction which can be computer read is the expression from one person in 
the medium according to claim 20 which can be computer read, and constitutes the expression of a code book is a 
medium which is characterized by transforming a different expression to a computer and which can be computer 
read. 

[Claim 22] It is the medium which is characterized by transforming expression to a computer and which can be 
computer read by transmitting to the receiver with which said instruction which can be computer read can 
reconstruct expression for at least one index of a synthetic expression in the medium according to claim 20 which 
can be computer read. 

[Claim 23] It is the medium which is characterized by said instruction which can be computer read transforming the 
expression of a face to a computer in the medium according to claim 20 which can be computer read and which can 
be computer read. 

[Claim 24] The code book which has stored the data which are the code book embodied on [ which can be computer 
read ] the medium, and define the 1st face expression set of the 1st person. It is data embodied on [ which can be 
computer read ] the medium, and these data define the 2nd face expression set. This 2nd face expression set Face 
expression conversion system characterized by having the data used as a training expression set of the 2nd 
different person from said 1st person, said training expression set, and the conversion processor that draws a 
transform function from the expression which corresponds from said 1st expression set. 

[Claim 25] It is the face expression conversion system characterized by said conversion processor having a linear 
transform processor in face expression conversion system according to claim 24. 

[Claim 26] This synthetic expression set is face expression conversion system characterized by being drawn by 
applying a transform function to code book expression including the synthetic expression set further embodied in 
face expression conversion system according to claim 24 on [ which can be computer read ] the medium. 
[Claim 27] It is the face expression conversion system characterized by said transform function compensating the 
size of the face of the 1st person and the 2nd person, and the difference in a configuration in face expression 
conversion system according to claim 24. 

[Claim 28] In face expression conversion system according to claim 24 said conversion processor n expression 
vector sets of the face which expresses each expression as a 3m vector containing x in the correlation sample 
location of m pieces, y, and z variation rates, and is changed are gal... as n The vector set to which a target face 
corresponds is gb1... When given as n. they are 3m linearity least square system of the following format, and 
[Equation 2]. 



Face expression conversion system characterized by drawing a transform function by **** Lycium chinense by 
calculating one linear prediction child set aj for every coordinate of ga. 

[Claim 29] When the face lighting system which offers the light source from which plurality differs simultaneously 
since the face of a photographic subject is irradiated, and said face lighting system irradiate, They are the data 
incorporation system which incorporates both structure data and reflection factor data from the face of a 
photographic subject, and the 1st composition expression code book. This 1st composition expression code book 
The training expression set obtained from the photographic subject is received. Said training expression set. The 
transmitter which has the 1st composition information code book compounded by calculating a transform function 
using a corresponding non-compounded code book expression, and applying said transform function to all the 
expression in the 1st composition information code book, The reconstruction module for being the receiver linked 
possible [ this transmitter and a communication link ]. and reconstructing a face image. It has the receiver which has 
said 1st composition information code book and the 2nd composition information code book which has stored the 
same synthetic expression. Said transmitter Or it corresponds, in order to incorporate an additional expression of a 
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photographic subject, and to look for the expression which agrees mostly, the 1st code book is searched. Face 
expression conversion system characterized by transmitting the index of the code book expression which agrees 
mostly or it corresponds to said receiver for the face image reconstruction by the reconstruction module. 
[Claim 30] It is the face expression conversion system characterized by said lighting system including at least one 
polarization light source in face expression conversion system according to claim 29. 

[Claim 31] It is the face expression conversion system characterized by said lighting system including two or more 
polarization light sources in face expression conversion system according to claim 29. 

[Claim 32] It is the face expression conversion system characterized by including the patternizing light source 
constituted so that said lighting system might project a pattern on the face of a photographic subject in face 
expression conversion system according to claim 29. 

[Claim 33] It is the face expression conversion system characterized by including the infrared light source to which 
said lighting system projects a pattern on the face of a photographic subject in face expression conversion system 
according to claim 29. 

[Claim 34] The light source from which said plurality differs in face expression conversion system according to claim 
29 is face expression conversion system characterized by all being the infrared light source. 
[Claim 35] How to animate the description of the face characterized by being the approach of animating the 
description of a face, defining the fragmentation side approximated to the geometry of the face from which plurality 
differs, and fitting the same fragmentation side as each of two or more faces. 

[Claim 36] The approach characterized by said step to define defining a fragmentation side with rough mesh 
structure in an approach according to claim 35. 

[Claim 37] It is the approach characterized by rough mesh structure having a triangular mesh in an approach 
according to claim 36. 

[Claim 38] The approach characterized by performing continuation optimization operation covering the top-most- 
vertices location of a fragmentation side in an approach according to claim 35 at said step to fit. 
[Claim 39] The approach characterized by fitting a fragmentation side to a face, without changing the connectability 
of the mesh which defines the fragmentation side by said step to fit in an approach according to claim 35. 
[Claim 40] The approach characterized by making the smoothing functional relevant to the mesh which defines the 
fragmentation side by said step to fit into the minimum in an approach according to claim 35. 
[Claim 41] The approach characterized by choosing one or two constraint or more relevant to the mesh which 
defines the fragmentation side by said step to fit in an approach according to claim 35, and fitting said constraint 
directly on the point with which it corresponds on a field. 

[Claim 42] It is the approach characterized by relating said constraint with one of an eye, a nose, and the openings 
in an approach according to claim 41. 

[Claim 43] The approach characterized by making the functional which includes the term which expresses distance, 
smoothness, and constraint with said step to fit in an approach according to claim 35 into the minimum. 
[Claim 44] The approach characterized by calculating the solution of a series of linearity least square problems in an 
approach according to claim 35 at said step to fit. 

[Claim 45] One or two media or more which are characterized by storing in one or the two computers or more 
concerned the instruction which realizes an approach according to claim 35, and which can be computer read when 
it performs by one or two computers or more and which can be computer read. 

[Claim 46] How to animate the description of the face characterized by being the approach of animating the 
description of a face, defining the fragmentation side approximated to the geometry of the face from which plurality 
differs, fitting the same fragmentation side as each of two or more of said faces, and changing the expression of one 
face into the expression of the face of another side for a face and a face using matching and matching with a face 
and a face. 

[Claim 47] Are the approach of animating the description of a face and three-dimension data are measured about 
the face from which plurality differs. The general-purpose face model used since a corresponding face model is 
acquired and it maps to each corresponding face model is defined only one. Are directly mapped by the point with 
which it corresponds on each corresponding face model. Two or more points on said general-purpose face model 
are chosen, and this general-purpose face model is fitted to each of a corresponding face model. At the step 
concerned to fit How to animate the description of the face characterized by mapping each of the selected point 
directly on the point with which it corresponds on each corresponding face model. 

[Claim 48] It is the approach which said step to define defined the fragmentation side from base mesh structure, and 
said fragmentation side has approximation-ized geometry of a face model in an approach according to claim 47 
including two or more top-most vertices, and is characterized by operating only the location of the top-most 
vertices of a fragmentation side at said step to fit 

[Claim 49] The approach characterized by operating the base mesh which defines the fragmentation side by said 
step to fit in an approach according to claim 47. 

[Claim 50] The approach characterized by operating the base mesh which defines the fragmentation side by said 
step to fit in an approach according to claim 47, without changing the connectability of a base mesh. 
[Claim 51] The approach characterized by measuring three-dimension data using a laser range scan in an approach 
according to claim 47 at said step to measure. 



[Translation done.] 
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* NOTICES * 

JPO and NCIPI are not responsible for any 
damages caused by the use of this translation. 

1. This document has been translated by computer. So the translation may not reflect the original precisely. 

2. **** shows the word which can not be translated. 
3.1n the drawings, any words are not translated. 

DETAILED DESCRIPTION 

[Detailed Description of the Invention] 
[0001] 

[Field of the Invention] This invention relates to the approach and system of the modeling for animating a face with 
reality, and a rendering. Specifically, this invention relates to the approach and system for a face image processing. 
[0002] 

[Description of the Prior Art] In the field of computer graphics, the rendering of various objects is carried out and 
the object is displayed from a computer display for a user. For example, it is typical that the computer graphics 
application for generating and carrying out the rendering of the computer object in a computer game, and displaying 
on a computer screen or television is included. The modeling and rendering of an image with a real feeling serve as 
an efforts target without an intermission for what is engaged in the computer graphics field. The field which serves 
as an efforts target especially in the computer graphics field is carrying out the rendering of the face image with a 
real feeling. As one example, carrying out the rendering of the display of the individual who has participated in 
conversation is performed with a certain specific computer graphics application. By computer, the image with which 
the rendering of what is often seen was carried out eventually [ of this individual ] is an image by which the 
rendering was carried out, and differs from the actual individual often substantially. 
[0003] 

[Problem(s) to be Solved by the Invention] It is a difficult activity especially by two reasons that carry out the 
modeling of a face and face expression with a real feeling, and they carry out a rendering. One is because human 
being's skin has an unsuitable reflection factor property in carrying out modeling with various shading models 
available now. For example, the well-known Phong model is inadequate for carrying out the modeling of human 
being's skin. When carrying out the rendering of the face expression and another has a gap also a thing and a little 
which is sensed as a motion of a "real" face, the gap is because even an indifferent observer is sensed as the not 
right thing. If the existing motion incorporation (capture) system of a face is used, considerably persuasive face 
animation can be created, but when the incorporated motion is applied to another face, it has often happened that 
the persuasive power declines far and becomes very strange. For example, if it takes into consideration that the 
face which became the origin of the motion is the same face, it is not difficult to animate a motion of each face, if 
the sampling of a motion of the face is taken from a certain person. Consequently, the description of the motion 
which is the same between expression other than a certain expression, or is dramatically similar will be acquired. 
However, it has often happened that it is weak to change a motion of this person's face into another person's face 
since the difference (for example, the size and the configuration of a face) in a proper is between these two faces. 
[0004] Therefore, this invention makes it a technical problem to offer the amelioration system and approach for 
carrying out the modeling of the texture and reflection factor of the skin of human being, and is made. Moreover, by 
changing a motion of a certain person's face into a motion of the face of an another person object this invention 
makes it a technical problem to offer the system and approach for carrying out the reuse of the incorporation 
(capture) data of a motion of a face, and is made. 
[0005] 

[Means for Solving the Problem] At the operation gestalt currently illustrated and explained below, it is a three 
dimension. (3-D) The data which have described the countenance are incorporated, a motion of a face is changed 
into another individual from a certain individual with reality, and the new technique for carrying out the modeling of 
the reflection factor of the skin is proposed. 

[0006] With the operation gestalt currently explained here, human beings photographic subject is prepared, and 
since the face of a photographic subject is irradiated, the light source from which plurality differs is used, the 
structuring light source to which one in two or more light sources projects a pattern on the face of a photographic 
subject (structured light source) it is . It becomes possible to incorporate the data one or two cameras or more 
have described the countenance of 3-D of a photographic subject to be according to this structuring light source. 
Although other one more light source is prepared further, since the face of a photographic subject is irradiated, this 
is used. The face of a photographic subject is a reflection factor property variously by the other one light sources, 
(reflectance properties) It checks. Moreover, the light source of this another side is a polarizing filter, (polarizing 
filter) It is used together and is the specular reflection component of the reflection factor of a face, (specular 
component) It is removed. Namely, diffuse reflection component (diffuse component) It is incorporated with a 
camera. If the light source from which plurality differs is used, it will become possible to check simultaneously both 
the structure property of a face, and a reflection factor property. When the light source is chosen carefully, the light 
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source is made into a narrowband and it is an ac(justment narrowband filter, for example, (matching narrowband 
filter) If it is used with a camera, it will become possible to remove the effect of the source of perimeter lighting. 
[0007] Two useful items are obtained as a result of the lighting process currently explained here. Namely, (1) A 
range map (or depth map (depth map)) and the pattern of (2) structuring light source are the images of the face 
which is not projected there. Field normal [ as opposed to / 3D side is drawn from a range map and / 3D side ] 
(surface normal) It is calculated. In order to define 3D side, when processing a range map and the filtering step as an 
option is included in the processing, it is a general-purpose face template, (genericface template) It will be combined 
with a range map and an unnecessary noise will be refused. The image of the calculated field normal and a face is an 
albedo map. (albedo map) It is used in order to draw, an albedo map — special type texture map (texture map) it is - 
- each sample has described the diffuse reflection factor of the face in the specific point on a field (point) there. 
Therefore, the information which has described 3D side (that is, field normal) of a face at this event of a process, 
and the information which has described the reflection factor (that is, albedo map) of a face will be checked. 
[0008] With a certain operation gestalt the information or data obtained in the lighting process is used in order to 
change the expression of a certain person's face into the expression of the face of an another person object. At this 
operation gestalt, it is a code book, (code book) The view is taken in and used. 

[0009] Code books are much general-purpose expression of an another person object (person A), (generic 
expressions) The described data are stored. One target is changing the expression of an another person object 
(person B) for the expression of a code book using ejection and its expression. In order to perform this, by the 
approach by this invention, it trains using Person B. Expression (training expressions) The set is created. Training 
expression consists of sets of the expression which exists in a code book. It is a transform function when training 
expression and the code book expression corresponding to each expression are used, (transformation function) It is 
drawn. This transform function is the set of a synthetic expression which should have agreed on Person's B 
expression, (a set of synthetic expressions) It is used in order to draw. That is, when a transform function is drawn, 
this is applied to each of the expression in a code book, and it is made for code book expression to agree on 
Person's B expression. Therefore, if a synthetic code book expression is searched when a new expression which 
may not exist in a training set is received from Person B, the expression which agrees most on Person's B 
expression can be found. 

[0010] Common face structure which can use the expression of a face, and a motion with another operation gestalt 
in order to change into another face from a certain face (common face structure) It defines, the operation gestalt 
currently explained here — common face structure — rough mesh structure (coarse mesh structure) Or 
fragmentation side used as a foundation in case it consists of "a base mesh (base mesh)" and this changes a 
certain person's expression into another person's expression (subdivision surface) The definition is given. A 
common-base mesh is used to all faces, and 2 or three or more faces are made to be matched. Therefore, if the 
structure defined by this is used, it will become possible to be adapted for an another person object from a certain 
person in a motion of a face. According to this operation gestalt, the technique for being adapted for the face model 
of a photographic subject in a fragmentation side is used. By the technique by this invention, some points on a 
fragmentation side are defined as being directly mapped by the point with which it corresponds on a face model. 
This is the same also about all different face models that may happen. This constraint (constraint) When it adds, a 
base mesh will have the property of fitting a different face model similarly. Furthermore, smoothing functional made 
into the minimum with the algorithm by this invention so that a base mesh and a face model might certainly be 
matched (smoothing functional) It is used. 

[0011] With another operation gestalt the reflection factor processing technique which metrizes the reflection 
factor of the face of a photographic subject is prepared. At the technique according to this invention in order to 
measure a reflection factor, a reflection factor is a diffuse reflection component, (diffuse component) Specular 
reflection component (specular component) It dissociates and processing of a diffuse reflection component has 
taken the lead. 

[0012] It is an albedo map in order to measure a diffuse reflection component (albedo map) It defines first This 
albedo map is defined by preparing first a camera and the photographic subject irradiated by the light source from 
which plurality differs. The light source is a polarizing filter, (polarizing filter) It is filtered. This filtering is used 
together with the polarizing filter with which the polarizing filter was placed in front of the camera, and is performed 
by preventing that inhibit specular reflection or specular reflection is recorded. An image sequence is photoed so 
that the head of a photographic subject may be surrounded. Each individual image is processed so that the individual 
albedo map corresponding to the image may be obtained. All the albedo maps to a specific photographic subject are 
combined so that a single albedo map may be obtained to the whole face of the photographic subject 
[0013] 

[Embodiment of the Invention] Hereafter, the operation gestalt of this invention is explained to a detail with 
reference to a drawing. 

[0014] In order to carry out the rendering of the face expression to a face with an outline real feeling, a very exact 
model needs to express the reflection factor of the skin, and a motion of a face. Below, the measurement data 
about geometry, a motion, and a reflection factor is used, and the approach and technique for carrying out the 
modeling of the face, animating, carrying out a rendering, and reproducing the appearance and face expression of a 
face of a specific person with reality are explained. Since the perfect model containing geometry and bidirectional 
reflectance is built, the rendering of the face can be carried out in any lighting conditions and display conditions. 
Structuring face model with which between different faces was matched by the modeling system explained below 
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and the approach (structured face model) It is built and this face model serves as a foundation of animation 
actuation of various faces. 

[0015] The operation gestalt by this invention has each part and relation which constitute a face modeling process 
so that it may explain below. In order to set the geometry expression as the foundation of a face model and to make 
it become the face modeling of many henceforth, and the foundation of rendering actuation by structuring and 
unification-izing the expression of geometry, at the operation gestalt of this invention, it is the conventional field 
adaptation technique, (surface fitting technique) It extends and is a general-purpose face, (generic face) He is trying 
to make a different individual's face agree. Although the latest rate measurement of a cutaneous reflex is first used 
in order to build a reflection factor model with a real feeling, this is well-known and is put in practical use. 
Furthermore, improved texture incorporation process (texture capture process) It is used and the diffusion texture 
map measured newly is added. General-purpose mesh (generic mesh) In order to animate, the improved technique is 
used and the field configuration of having been suitable for the high quality rendering is made to be acquired. 
[0016] The outline of the example of the desktop computer 130 which can be used with the operation gestalt 
currently explained to drawing 1 here as instantiation computer system preliminary knowledge is shown. Under the 
situation of a distributed computing environment, various things of a computer like a graphic display are usable. 
These computers carry out the rendering of the graphics according to the explanation indicated below, and they can 
be used for them in order to process an image (image). 

[0017] The computer 130 includes one, two processors or more or the processing unit 132, the system memory 134. 
and the bus 136 136. i.e.. the bus which has combined the various system components containing a system memory 
134 with the processor 132. Although the bus 136 is expressed with any one of the several sort type bus structures, 
or two or more, in this, the memory bus which adopted various bus architectures or a memory controller, the 
peripheral bus. the high-speed graphics port and the processor, or the local bus is contained. As a system memory 
134. it is a read-only memory, (read only memory-ROM) 138 and random access memory (random access memory- 
RAM) There is 140. Basic input/output system (BASIC input/output system-BIOS) 142 consists of basic routines 
which support the information transfer between the elements in a computer like [ at the time at the time of a start- 
up ]. and is stored in ROM 138. 

[0018] The computer 130 has equipped an optical disk drive 150 still like the hard disk drive 144 written between 
hard disks (not shown), the magnetic disk drive 146 written between the dismountable disks 148, and CD-ROM and 
other optical media written between the dismountable optical disks 152. The hard disk drive 144, the magnetic disk 
drive 146, and the optical disk drive 150 are connected to the bus 136 through the SCSI interface 154 or other 
kinds of a certain of applicable peripheral interface. These drives and the medium of those which can be related 
computer read store the instruction which can be computer read, DS, a program module, and other data as non- 
volatile storage for a computer 130. in the instantiation environment where it is explaining here, although the hard 
disk, the dismountable magnetic disk 148. and the dismountable optical disk 152 are adopted, if it is this contractor. I 
will be understood — as — a magnetic cassette, flash memory card, a digital videodisc, and random access memory 
(RAM) Read-only memory (ROM) etc. — it is also possible to use the medium of other types which can store the 
data which can access a computer like and which can be computer read in the operating environment of 
instantiation. 

[0019] Some program modules can be stored in a hard disk 144, a magnetic disk 148, an optical disk 152, and 
ROM138 or RAMI 40, and operating systems 158 and 1 or two or more application programs 160, other program 
modules 162, and the program data 164 are contained in it. A user can let input devices, such as a keyboard 166 and 
a pointing device 168, pass, and can input a command and information into a computer 130. A microphone, a joy 
stick, a gamepad, a satellite dish, a scanner, one, or two cameras or more are possible as other input devices (not 
shown). The above and other input devices are connected to the processing unit 132 through the interface 170 
combined with the bus 136. The monitor 172 or the display device of other types is also connected to the bus 136 
through the interface of a video adapter 174 etc. As for a personal computer, it is typical to have equipped other 
circumference output devices (not shown) like a loudspeaker or a printer other than a monitor. 
[0020] Usually it operates by the networking environment which uses a logic connection with one or two remote 
computers [ like a remote computer 176 ] or more whose computer 130 is. another personal computer as a remote 
computer 176. a server, a router. Network PC, and a pier — although it is possible to make it a device or other 
common network nodes and only the memory storage device 1 78 is shown in drawing 1 . it is typical for there to be 
many elements mentioned above in relation to the computer 130, or to have equipped ail. As a logic connection who 
shows drawing 1 , it is a Local Area Network, (local area network-LAN) 180 and wide area network (wide area 
network-WAN) There is 182. This kind of networking environment is office, the computer network in an enterprise, 
intranet and the Internet, (the Internet) It is the then common existence. 

[0021] When used by LAN networking environment, the computer 130 is connected to the local network 180 through 
the network interface or the adapter 184. When used by WAN networking environment in order to establish 
communication on a wide area network 182 like the Internet, as for a computer 130, it is typical to have equipped 
other means like a modem 186 or a network interface. A modem 186 has built-in and an external mold, and is 
connected to the bus 136 by both through the serial port interface 156. At networking environment the program 
module mentioned above in relation to the personal computer 130 or its part can be put on a remote memory 
storage device. The network connection of a graphic display is instantiation and it is also possible to use other 
means to establish the communication link between computers so that I may naturally be understood. 
[0022] Generally, the data processor of a computer 130 is programmed with the instruction of a computer variously 
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stored in the medium which can be computer read by differing in time amount, respectively. As for a program and an 
operating system, being decentralized on a floppy disk or CD-ROM is typical. A program and an operating system 
are installed or loaded to the secondary memory of a computer from a floppy disk or CD-ROM. At the time of 
activation, as for a program and an operating system, the part is loaded to the primary electro nick memory of a 
computer at least the type which mentioned above the instruction or program for realizing the step currently 
explained below in relation to a microprocessor or other data processors according to this invention currently 
explained here, or others — many things are stored in the storage of a type which can be computer read. Moreover, 
according to this invention, the computer itself is programmed according to the approach and technique of being 
explained below. 

[0023] Like the expedient top of explanation, and an operating system, although a program and other executable- 
program components are shown as a discrete block here, these programs and components differ in a stage, are put 
on the different-species storage component of a computer, and are performed by the data processor of a computer 
so that I may naturally be understood. 

[0024] Conventionally [ instantiation system ] for incorporating the structure and the property of the face, it is an 
incorporation system, (capturing system) Both the face structure of the whole face and a reflection factor property 
were not able to be independently incorporated simultaneously from mutual. For example, structuring light 
(structured light) These systems are reflection factors although the system which uses it and incorporates the 
structure of a face exists, (reflectance) The property [ like ] of a face is not incorporated. As for these systems, the 
structure of a face is not incorporated although similarly the system which incorporates the reflection factor of a 
face exists. If the structure and the reflection factor of a face can be incorporated simultaneously independently 
mutually, another actuation will be added to collected data and it will become possible to use for various face 
renderings and animation actuation. Although one example of instantiation rendering actuation is explained, the 
information or data obtained as a result of the system and approach of lower-**(ing) can be used in other various 
fields below so that I may naturally be understood. For example, construction of the face model which has a real 
feeling aiming at animation, such as recognition of the face aiming at security, the exchange between personal users, 
etc., a game, and a movie (a film, animation), as an application field, and a user incorporate their face easily, and are 
made to be able to use it by interactive entertainment or business communication. 

[0025] Drawing 2 shows the instantiation system 200 which is suitable for using it when incorporating simultaneously 
the face structure and the reflection factor property of a face of a photographic subject. This instantiation system 
equips the data incorporation system which carried out the form of one or two cameras or more, and one of them is 
illustrated as a camera 202. A camera 202 equips a CCD image sensor and a related circuit, operates a CCD array, 
reads an image in a CCD array, changes an image into a digital form, and makes it possible to notify the image to a 
computer. Moreover, a system is a face lighting system which carried out two or more light sources or the form of a 
projector, (facial illumination system) It has equipped. It is a JIEN lock so that these cameras may make 
simultaneous incorporation possible in time, when two or more cameras are used, (genlock) It is carried out. The two 
light sources 204 and 206 are used in the example of a graphic display. The light source 204 is a structuring pattern 
(structured pattern) projected on the face of a photographic subject desirably. It outputs. If the light source 204 is a 
suitable location, it is possible to place anywhere. This pattern makes it possible to incorporate the structure 
information or data about 3D configuration of a face of a photographic subject with a camera 202. If the pattern 
which consisted of light of an infrared field is adopted, it is advantageous, but if it is the suitable light source, it is 
usable at every light source. The light source 206 outputs desirably the light which enables it to incorporate the 
diffuse reflection component of the reflection factor property of a face with a camera 202. The light source 206 is 
for example, a beam split, (beam splitting) If it puts on lens 202a of a camera, and the same rank by technique, it is 
advantageous, but placing anywhere is possible if it is a suitable location. As for this light source, it is possible to 
also make it adapted so that the lens of a camera may be surrounded. Moreover, this light source is the specular 
reflection component of a reflection factor, (specular component) It is chosen so that it may be inhibited or 
removed. At the example of a graphic display, it is a linearity polarizing filter, (linear polarizing filter) Since it is 
adopted in order to obtain polarization lighting, and another polarizing filter has sense which intersects 
perpendicularly with said linearity polarizing filter and it is placed before lens 202a, the specular reflection from a 
face is made not to be recorded with a camera. The lighting system mentioned above is simulated using the light 
source of a different frequency, for example, the light source corresponding to the red and the green channel of a 
camera. In addition, both of these channels can be made an infrared field. Furthermore, if it chooses so that the light 
source may become a narrowband (for example, 780-880 nm), the effect of an ambient light is removable. It is only a 
time of being filtered so that a camera may also become a narrowband that this property is attained. Since an 
ambient light is diffused to the lighting from the light source being concentrated on the wavelength of a narrowband 
covering the wavelength of a broadcloth range, the light from the light source becomes more powerful than the 
ambient light at the time of the above wavelength. Since a camera is filtered so that only the wavelength emitted 
from the light source may be recorded, being influenced by the ambient light does not almost have it. Consequently, 
a camera will detect only the effect the selected light source affects a photographic subject. 
[0026] If the light source from which plurality differs is used, if the infrared light source is used together with the 
polarization light source (this can also be made the infrared light source), specifically, it will enable a camera to 
incorporate independently the structure information about a face, the reflection factor information about data and a 
face (from the light source 204), or data (from the light source 206) simultaneously mutually. Reflection factor 
information has described the diffuse reflection factor property of a face to structure information having described 
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the three-dimension modality of a face. When this information is processed by computerization image processors, 
such as a computer 208, the information or data which can be used by future face animation actuation will be 
obtained. At the example explained below, this information is three-dimension data (3D data) and an albedo map. 
(albedo map) It contains. 

[0027] Drawing 3 is a flow chart which has indicated the step of the approach by the operation gestalt explained 
here. According to the approach explained here, the information or data about the structure and the reflection 
factor property of a face are collected simultaneously, and it is possible to process. It irradiates by the light source 
from which, as for the face of a photographic subject, plurality differs at step 300. The instantiation system which 
realizes this step is shown in drawing 2 . In addition, although the two instantiation light sources are used in the 
example of a graphic display, it is also possible to use two or more of other light sources so that I may naturally be 
understood. At step 302, range map data (depth map data) and image data are measured from the exposure of step 
300. That is, it will be used, in order that a camera may be enabled to detect the rate of a light reflex and it may 
obtain both image data (that is, reflection factor) in which this rate of a light reflex does not contain the structure 
light source pattern range map data and there by the exposure of step 300. Range map data and image data are 
passed to a computer 208 ( drawing 2 ), and are processed there. At this event, it is step 304. and is a general- 
purpose face template, (generic face template) This is an option, although the various noises produced in relation to 
range map data are removable if it applies to range map data. A general-purpose face template can be considered to 
be 3D filter from which the noise contained in range map data is removed. A general-purpose face template is 
obvious to the familiarity person of this field. 

[0028] At step 306, range map data are used, and 3D side is drawn or calculated. If it is a suitable algorithm, it is 
usable with every algorithm and this is obvious to the familiarity person of this field. There are some which are 
indicated by the following paper as an example of an algorithm, namely. Turk & Levoy work "zipper DOPORI gon 
mesh (Zippered Polygon Meshes) from a range image" (SIGGRAPH94) and F.Bernardini. J.Mittleman. H.Rushmeier. 
C.Silva. and G.Taubin collaboration "baud ruby BOTTINGU algorithm (The Ball-Pivoting Algorithm) for field 
reconstruction" (Trans.Vis.Comp.Graph.5:4 (1999)) it is . At step 308, the field normal vector ("field normal") to 3D 
side of step 306 is calculated using a well-known algorithm. One method of performing this task is calculating the 
normal to a triangle, taking the average of the triangle normal of the circumference of each top-most vertices, 
making a top-most-vertices normal, and interpolating a top-most-vertices normal so that the interior of three 
square shapes each may be crossed. Although it is natural, it is also possible to use other approaches. Next, at step 
310, the image data of the field normal calculated at step 308 and step 302 is used, and an albedo map is drawn. An 
albedo is a special type texture map and each sample has described the diffuse reflection factor of the face in the 
specific point on the face there. Derivation of an albedo map is obvious to the familiarity person of this field, if the 
information shown above is given. The algorithm of instantiation is Marschner work "reverse rendering (Inverse 
Rendering for Computer Graphics) of computer graphics" (PhD thesis, Cornel University, and August 1998). It is 
indicated. 

[0029] At this event as shown in drawing 2 , 3D data which have described the structure description of the face of 
a photographic subject, and the albedo map data which have described the diffuse reflectance of a screen are 
obtained as a result of lighting processing. 

[0030] Since the above-mentioned lighting processing can be used in order to extract the information described, the 
extracted information can be used for the suitable object. With the operation gestalt with especially an advantage, 
the extracted information is used in order to extract and recognize the expression of a photographic subject. This 
information can be used for expression conversion. With the operation gestalt by this invention currently explained 
immediately in the back, a certain person's expression can use the expression of an another person object in order 
to have and change a real feeling. 

[0031] At the expression conversion **** expression conversion implementation gestalt by the activity of a code 
book, it is a code book, (code book) A view is taken in and it is used by expression conversion actuation in which it 
is explained below. Drawing 4 shows the example of the code book 400 which has stored a different expression of a 
large number incorporated by a certain person. Such expression is general-purpose expression, (generic 
expressions) That is, the expression from a general-purpose person can be considered from a specific individual. In 
this example, expression has the range from expression 1 to Expression N. Expression 1 is a "smile", expression 2 is 
at a "wry look" and expression 3 is the expression of the "resentment" Hereafter, it is the same. A thing like the 
process which it is described in the form of those geometry and explained mathematically immediately in the top to 
incorporate by the suitable approach is possible for the expression stored in the code book 400. 
[0032] In order to perform expression conversion, a part of expression in a code book 400 is used, and it is a 
transform function, (transformation function) It is drawn first In order to draw a transform function, it is a training 
expression set. (training set of expressions) The view 402 is taken in. The expression training set 402 consists of 
sets of the expression obtained from an individual with the expression other than the individual described by the 
code book 400. The training expression of the training set 402 is the subset (subset) of code book expression. That 
is, each expression in a training set supports the expression in a code book 400. For example, the training set 402 
can be constituted from three expression 1, i.e., expression, expression 2, and expression 3, and expression has 
become a "smile", a "wry look", and the "resentment" there, respectively. The target of a transform function 
applies the geometric deformation related with the expression of a training set to reception, applies the geometric 
deformation to all the expression of a code book 400, and is making it code book expression serve as an expression 
expression with a real feeling. That is. it is thought that the face of an everybody object carries out the method of 
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deformation that geometry differed, also with the expression of which arbitration. When deformation of a certain 
person's geometric face at the time of a certain expression is simply applied to the face of an another person object 
in order to carry out the rendering of the expression, the face to which the deformation was applied will seem to 
have been distorted dramatically. The geometry of a face not only differs, but such a thing happens, because 
deformation of a face differs like [ between faces ]. Therefore, a transform function with which the best conversion 
to another expression set from a certain expression set is obtained is drawn. 

[0033] If it returns and explains to drawing 4 , the linear transform processor 406 is shown in drawing 4 . This 
conversion processor 406 can be realized in suitable hardware, software, firmware, or such combination. The linear 
transform processor 406 is realized by software in the example of a graphic display. A linear transform processor 
receives the training expression set 402 and the corresponding code book expression 404 as an input. A conversion 
processor processes these inputs and draws a transform function 408. When this transform function 408 is applied 
to all the expression in a code book 400. it is a synthetic expression set. (synthesized set of expressions) It is 
obtained. This synthetic expression set expresses the expression of a code book operated according to the 
geometric deformation relevant to the expression of the person by whom the training expression set was got. 
[0034] The variation rate of a face when expression is the same is not the same in a person who is different by the 
following two reasons. It is because one of them is not in agreement with accuracy if the motion incorporation 
sample point (it is shown by this example according to this specific algorithm how a motion of a face can express) 
has an error in a location. Another is because the configuration and size of the head change between persons. 
[0035] The 1st inequality is cancelable by all faces' moving, incorporating and re-sampling displacement data by the 
fixed location set on a general-purpose mesh. This is explained to the part of the following "instantiation systems 
for building a face model and approaches" in detail. There, the fixed location set is called "the correlation sample 
location (standardsample positions)." a re-sampling function — mesh deformation function (mesh deformation 
function) it is . Correlation sample locations are top-most vertices of the face mesh corresponding to top-most 
vertices of the general-purpose mesh subdivided once. 

[0036] It is necessary to change displacement data into another face from a certain face, and to compensate the 
size of a face, and change of a configuration with the 2nd inequality. In the example of a graphic display, this is 
performed by finding the best linear transform from one side to another side, after finding the small training set of 
the expression which corresponds about two data sets. The following case is considered as one example. That is, in 
the experimental environment the label was manually attached to 49 corresponding expression in which the 
expression showing feeling contains various strength of two or more expression. About a motion of voice. 10.000 
frames is time amount distortion, (time warping) Alignment was automatically carried out using technique. 
[0037] Each expression is expressed by the 3m vector g containing x in the correlation sample location of m pieces, 
y, and all the z variation rates. They are [ Every one set aj is calculated for every coordinate of gn by calculating the 
solution of 3m linearity least square system shown below. ] n expression vector sets of the face changed gal ... It is 
the vector set to which it is referred to as n and a target face corresponds gb1 ... When n, he is a linear prediction 
child, (linear predictors) 
[0038] 
[Equation 3] 



[0039] In the example of a graphic display, only the subset with each small point of gaj is used. Specifically, only the 
correlation sample point and the point which shares an edge are set as the object of consideration. In the used 
mesh, it is ******. (average valence) Since it is about 6, typical gaj is 18 elements. The system of the result is n x 
18 generally. 

[0040] The linear system of a result may not have enough conditioning and the linear prediction child aj is not 
conceptualized good in that case. Singular value (singular value) Diffusion is a false inverse function, (pseudoinverse) 
Since it is controlled when calculating and calculating the solution to. conceptualization is improved substantially. All 
less than alphasigmal singular values are made into zero, sigmal [ however, ] — the maximum singular value of a 
matrix — it is — alpha = 0.2 and ... it is 0.1. 

[0041] Drawing 5 is a flow chart which has indicated the step of the expression conversion approach by the 
operation gestalt currently explained here. The code book of expression is prepared at step 500. The example of this 
kind of code book is shown above. A training expression set is prepared at step 502. As an example of 
representation, this training set is the set of the expression from a different person from the person by whom code 
book expression was got If the training expression set is appropriate, it can be incorporated by every approach. As 
one example, expression can be incorporated using a system as shown in drjjwjngj? . After a training expression set 
is prepared, a transform function is drawn at step 504 using a training set and a code book. One example of an 
approach which does this activity is as having mentioned above. Although it is natural, unless it deviates from the 
pneuma and the range of this invention, it is also possible to use other approaches, for example, neural network 
(neural network) etc. — also using various nonlinear conversion — basic expression (basis expressions) It is also 
possible to use a weighted sum. If a transform function is drawn, this is applied to all the expression in a code book, 
and the synthetic expression set which can be used as a foundation of future face animation actuation will be 
obtained, or it will be defined. 

[0042] Application drawin g 6 of instantiation shows a system 600. There, it is shown by one example how the 
expression translation process mentioned above is adopted, a system 600 — the transmitting-side computing 
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system 602. i.e.. a transmitter, and the receiving-side computing system 604, i.e.. a receiver. — containing — these 
— the Internet (the Internet) etc. — it connects so that it may communicate through a network 603. It is 
constituted so that a transmitter 602 may incorporate a certain person's expression, as this was mentioned above 
with reference to drawin g 2 including the lighting system 200 ( drawing 2 ). Furthermore, the transmitter 602 
contains the same code book 400 as what was mentioned above with reference to drawing 4 . In addition, the code 
book shall be compounded by the synthetic expression set here, as mentioned above. That is, the training 
expression set obtained from the person with the expression which the lighting system 200 incorporated is used, a 
code book is processed, and the synthetic expression set is obtained. 

[0043] It is constituted so that a face image may be reconstructed from the data with which the receiver 604 
received this from the transmitter 602 including the reconstruction module 606. Moreover, the receiver 604 contains 
the same code book 400 as it by the side of a transmitter 602. Here, the person who is in a transmitter 602 side is 
going to communicate with the person who is in a receiver 604 side. If a face is moved for a communication link of 
the person who is in a transmitter 602 side, the face expression and motion will be incorporated and processed by 
the transmitter 602. It is possible to incorporate face expression in this processing, to search a synthetic code book 
to it, and to include to find the agreement expression most approximated in a code book in it. If the agreeing 
expression is found in a synthetic code book, the index of the expression will be sent to a receiver 604, and the 
animated face will be reconstructed using the reconstruction module 606. 

[0044] Face conversion drawing 7 of instantiation is drawing showing some effectiveness of the expression 
conversion by the operation gestalt currently explained here. The picture (A) of the 1st line constitutes the 1st 
person's (person A) synthetic face, and three different expression is shown there. These pictures are as a result of 
the motion of a face by which Person A was incorporated. The motion of the 2nd person's (person B) face is already 
incorporated. The motion of a face by which Person B was incorporated is shown in the 3rd line (C). Here, 3D 
motion data are incorporated by measuring a motion of a dot, when some color dots are placed on a person's face 
and a person's face is deformed, but naturally this will be understood, if this field becomes a familiarity person. In 
addition, a motion of a face can be incorporated by the system and approach which were mentioned above. The 
motion by which Person B was incorporated is used as mentioned above, and Person's A expression is changed. The 
result of this actuation is shown in the 2nd line. All the expression of a these three sets picture is mutually in 
agreement. As shown in this drawing, even if the expression of the 1st line (A) and the 2nd line (B) is drawn from 
two dramatically different persons, on the whole, the expression of the 1st line (A) and the 2nd line (B) is not similar 
for the 2nd person's (the 3rd line (C)) original copy expression to being dramatically similar. 
[0045] The model of the face needed in order to make the instantiation system for building a face model and an 
image with an approach real feeling consists of two parts. The amount of [ of a model ] part I is related to the 
geometry (that is, configuration of the face) of a face. On the other hand, the amount of [ of a model ] part II is 
related to the reflection factor (that is, the color and reflection property of a face) of a face. This section explains a 
part for part I of this model, i.e., the geometry of a face. 

[0046] The geometry of a face consists of what added another field which expresses an eye with a hide skin 
surface. A hide skin surface is drawn by scanning the head in a laser range, and is expressed with this example in 
the fragmentation side with a displacement map. An eye is another model, alignment of this is carried out to a hide 
skin surface, it is merged, and the perfect face model suitable for a high quality rendering is obtained. 
[0047] At the 1st step which builds a mesh adaptation face model, the fragmentation side closely approximated to 
the geometry measured with the laser range scanner is created. The fragmentation side is defined from a rough 
triangular mesh by the example of a graphic display using the fragmentation rule of Loop. The fragmentation rule of 
Loop is Charles Loop work "smooth fragmentation side (Smooth Subdivision SurfacesBased on Triangles) based on 
a triangle" (PhD thesis, University of Utah, and August 1987). It is explained in detail. Furthermore, a fragmentation 
side is work "partition smooth side reconstruction (Piecewise smooth surface reconstruction)" (Computer Graphics 
(SIGGRAPH'94 Proceedings) pp.295-302 and July 1994) besides Hoppe. The Sharp edge of what is indicated, and 
resemblance is added and it defines. In addition, irregular crease mask (crease mask) It is not used. Furthermore, 
darts (dart) When the edge between crease top-most vertices (crease vertex) is subdivided, only the new edge 
which adjoins crease top-most vertices is marked as a Sharp edge. 

[0048] The fragmentation side over all face models is defined using a single base mesh, and it is changing so that 
only a top-most-vertices location may be adapted for the configuration of each different face. In the example of a 
graphic display, the base mesh with 227 top-most vertices and the triangle of 416 pieces has the whole face 
configuration, it is defined as detailing an eye and near a lip, and the most complicated geometry and the most 
complicated motion have appeared there. The aperture of opening is the boundary of a mesh, and the adaptation 
process period remains closing by connecting the location of the top-most vertices where it corresponds on the 
upper lip and a metastoma to one. There is an edge of the fraction marked on sharp fragmentation rules in this base 
mesh, these create a corner on both sides of the aperture of opening, and it is useful to obtaining the location when 
turning up the both sides of a nose. Since the crease is taken in only to the chain of at least three Sharp edges with 
the corrected fragmentation rule, this model does not have a crease in a front face. Isolated top-most vertices 
(isolated vertex) Limit normal defined clearly (limit normal) It is because it cannot have. 

[0049] Drawing 8 shows the example of the rough definition mesh (central drawing) used according to this example. 
If drawing 8 uses this rough mesh how and maps the same fragmentation control (**) mesh in the displacement 
fragmentation side of each face, it shows whether natural response relation is obtained between a certain face and 
another face. This side face is explained in detail below. 
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[0050] The process used in order to fit a fragmentation side to each face is based on the algorithm indicated by 
work "partition smooth side reconstruction (Piecewise smooth surface reconstruction)" (Computer Graphics 
(SIGGRAPH*94 Proceedings) pp.295-302 and July 1994) besides Hoppe. Fundamentally, the field suiting method of 
Hoppe can be said to be what consists of three phases. That is. they are topology mold presumption (phase 1). 
mesh optimization (phase 2). and partition smooth side optimization (phase 3). 

[0051] In a phase 1, the triangular mesh in which the unorganized point set is given to a strange field top or the 
neighborhood and which consists of many triangles relatively is built In this phase, the topology mold of a field is 
judged and initial presumption of geometry is obtained. A phase 2 begins from the output of a phase 1, reduces the 
number of triangular, and improves the adaptation to data. In this approach, it is making to optimize the energy 
function which models clearly a trade-off between the targets to compete mutually called expressional compaction 
and good-izing of adaptation into the technical problem. As a free variable in this optimization procedure, they are 
the number of the top-most vertices in a mesh, and its connectability. (connectivity) And that location exists. A 
phase 3 fits an exact and brief partition smooth fragmentation side by leaving the optimization mesh (partition 
linearity side) obtained in the phase 2. and optimizing the energy function which trades off simplicity and the 
compatibility to data also in this case. If optimization is performed in a phase 3. the number of the top-most vertices 
in a control mesh, its connectability, its location, and the number and location of the sharp description will change. 
The sharp autmatic detection of the description and the recovery in a field serve as an indispensable part of this 
phase. 

[0052] In this operation gestalt. the processing method differs from the approach indicated by Hoppe etc. at two 
points. Since the 1st does not need to change the connectability of a control mesh, it is that continuous 
optimization is performed only to a top-most-vertices location. Furthermore, constraint of the description is added 
with the smoothing term. 

[0053] In the example of a graphic display, an adaptation process minimizes the following functional. 

[0054] 
[Equation 4] 

E(v) = J£f (v,p) + A£s{v) + t*Et(y) 

[0055] In the above, v is the vector of all top-most-vertices locations, and p is the vector of all the data points 
from a range scanner. The inferior letter of the 3rd term expresses distance, a configuration, and constraint. A 
distance function Ed is the sum square distance from the range scanner point to a fragmentation side, (sum- 
squared distance) It measures. 
[0056] 
[Equation 5] 

lip 2 

£'(v,p) = £a'||/»-n(v,/w)| 

i=l 

[0057] Setting above, pi is the Ith range point and pi (v. pi). It is projection of the range point on the fragmentation 
side defined by the top- most- vertices location v. For weight ai. the eye direction of the scanner in pi is pi (v. pi). It 
is the Boolean term which makes the point which is not in agreement with the field normal which can be set 
disregarded. Furthermore, the point which is distant from the field beyond a certain distance is refused. 
[0058] 
[Equation 6] 

1 if (s(pi\ n(Il(v 9 pi))) > 0 and \\pi - Il(v, jn)\ < d o 



ai = 



0 otherwise 



[0059] It sets above and is s (p). The direction and n (x) which go to the view of the scanner in Point p It is the field 
normal which faces the outside in Point x. 

[0060] The smooth function Es carries out flattening of the control mesh locally. This function measures the 
distance to the average of each top-most vertices to adjoining top-most vertices. 
[0061] 
[Equation 7] 

fill deg(vy) l=1 

[0062] Top-most vertices vki are the acjjoining top-most vertices of vj. 

[0063] A constraint function Ec is the sum square distance to the target location set which corresponds from a 
constraint top-most-vertices set simply. 
[0064] 
[Equation 8] 
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[0065] In the above. Ai catches and defines the marginal location of the j-th top-most vertices from a control mesh, 
and the marginal location of top-most vertices ci is the linear function added to the 3D point dL Although it is also 
possible to apply constraint strictly by re-parameterization of an optimization variable, if this constraint is used as 
software, it will have become clear that a repeat is smoothly performed to the desirable local minimum. Constraint is 
chosen by the user in order to make the face description of a genera I -purpose mesh agree with the description to 
which it corresponds on the suiting specific face. In this example, constraint of about 25 thru/or 30 is used and it is 
concentrating on an eye, a nose, and opening. Drawing 9 indicates the point with which it corresponds on a face 
model (B) to be constraint of the fragmentation control mesh (A) of 900. 

[0066] E (v) Since pi and ai are not the linear functions of v, it is a nonlinear least square problem to make it the 
minimum. However, ai is kept constant and it is pi (v, pi). This can be made into a linearity problem if it 
approximation-izes with the combination of fixed linearity-control top-most vertices. Therefore, an adaptation 
process will be advanced as a sequence of a linearity least square problem, and projection of pi on ai and a field will 
be re-calculated in front of each repeat. A fragmentation marginal side is approximation-ized by the mesh in specific 
fragmentation level in order to perform these count. Suiting a face is performed by a small number of repeat (less 
than 30). and with progress of a repeat constraint is updated according to a simple schedule and goes. The updating 
begins from high [ lambda ] and low [ mu ], will lead optimization to approximation-ization of a very smooth face, will 
go. and will progress to low [ lambda ] and high [ mu ]. and data and constraint will suit closely by the final solution. 
Actual computation time is decided by computation time of pi (v. pi). 

[0067] In order to make the mesh for a rendering, a field is subdivided to desirable level and the mesh smoothly 
approximated to the configuration of a face is obtained. A variation rate is calculated by making the line normal to 
the field in the top-most vertices intersect the triangular side defined by the original copy scan for every top-most 
vertices. This is explained to work "displacement fragmentation side (Displaced Subdivision 

Surfaces)" (SIGGRAPH'OO Proceedings and July 2000) besides Lee. Although the field of the result is reproducing 
all the descriptions with which the original copy scan was conspicuous in the mesh whose triangle has decreased a 
little, this is because the triangle of base mesh of the field where a face is important has increased. Inclusion 
multiplex resolving basis function set which was defined as that which parameterized the field by that 
parameterization according to the expression of this fragmentation base (built-in set of multiresolution basis 
functions) It will be obtained, and further, since the description constraint is used at the time of adaptation, natural 
response relation will be obtained over all the faces that suit using this approach. If you use this structure by the 
animation of a face, it is convenient in many respects. 

[0068] Drawing 10 is a flow chart which has indicated the step of the approach for building a face model according 
to the operation gestalt currently explained here. This approach can be realized in suitable hardware, software, 
firmware, or such combination. The approach is realized by software in this example. 

[0069] At step 1000. 3D data of one or two faces or more are measured, and a corresponding face model is 
obtained. In the above-mentioned example, 3D data were obtained by scanning a face in a laser range. 3D data can 
also be obtained by other approaches so that I may naturally be understood. The general-purpose face model used 
since one or two face models or more are suited is defined by step 1002. A general-purpose face model has the 
advantage that it can use so that the face from which a large number differ may be suited so that I may naturally be 
understood: Therefore, this serves as an improving point compared with the conventional approach that such a thing 
was not performed. At the example mentioned above, a general-purpose face model is a rough triangular mesh, 
(coarse triangle mesh) It has mesh structure which carried out the form. This triangular mesh defines the 
fragmentation side closely approximated to the geometry of a face. The fragmentation side about all the face models 
is defined by the example of a graphic display using a single base mesh. At step 1004, each point or constraint on a 
general-purpose face model is chosen. These points or constraint are directly mapped by the corresponding point 
marked on the face model. [ each ] Mapping of the point of these each is similarly performed about each of a 
different face model of a large number which may happen. At step 1006. a general-purpose face model suits one or 
two face models or more. This step is performed by operating only the location of top-most vertices so that it may 
be adapted for the configuration of each different face. Since continuous optimization is performed only covering a 
top-most-vertices location, the continuity of a mesh is not changed into an adaptation process period. Furthermore, 
in this adaptation process, each point or constraint is directly mapped by the face model. Furthermore, smooth term 
(smoothing term) It is added, and this is minimized so that flattening of the control mesh may be carried out locally. 
[0070] The displacement fragmentation side as for which the eye has carried out additional **** expresses the 
configuration of the hide skin surface of a face very well. However, there are some other descriptions made 
desirable in order to obtain a face with a real feeling. The most important description in rt is an eye. Since a laser 
range scanner does not incorporate the suitable information about an eye, a mesh is reinforced by adding the eye 
modeled independently for the rendering. Since an eye and its motion are not measured from a specific person 
unlike other parts of a face model, the appearance of a real eye is not necessarily reproduced. However, existence 
of an eye and a motion are dramatically important for the overall appearance of a face model. 

[0071] If it is a suitable eye model, an eye can be modeled using the eye model. In the example of a graphic display, 
in order to build the model which consists of two parts, the commercial modeling package is used. The amount of 
part I is the model of an eyeball, the amount of part II is the model of the hide skin surface of the circumference of 
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an eye, and a part for a palpebra. an eye socket, and perimeter regions of face is contained in it (the amount of this 
part II will call it "the orbital surface (orbit surface)"). In order to use an eye as some whole face models, it is 
necessary to fit the orbital surface to the individual face which it is going to model, and to sew up two fields. This is 
performed at two steps. First, distortion is attached according to the weighting function with which two mesh was 
defined on the orbital surface, and it is made to appear simultaneously in the part where the eye socket overlapped 
the face. Next 2 sides are omitted by this cardiac ellipsoid of a pair, and it joins by stitching and they are carried 
out so that it may become one mesh. 

[0072] In addition, according to these operation gestalten, one of the descriptions with the advantage of the 
operation gestalt mentioned above is that the structure or the framework which can be used in order to change a 
certain person's expression into another person's expression is acquired. Suiting an individual's face in a general- 
purpose face model is restrained so that every part of a general-purpose model may always be mapped by the same 
description on the face of an everybody object. For example, the left corner of opening in a general-purpose model 
is always mapped by the left corner of the opening by every person's face. If the set of an adaptation face model is 
used, in every face, the point corresponding to the specific point in a certain specific face can be judged. For 
example, suppose that the motion of the left corner of opening on Person's A face was measured. If it can judge 
which point of a general-purpose model is equivalent to the measure point if adaptation of the general-purpose 
model to Face A is used, next adaptation of the general-purpose model to Face B is used, it can judge which point 
of the face of B is equivalent to the point with which it was calculated on the general-purpose model, therefore 
whether the measure point of Face A is supported. Since it is necessary to know it is necessary which part of a 
face new in order to reproduce the point set lost motion on the measured face to move, this information is 
indispensable when changing a motion into another face from a certain face. 

[0073] A motion of the motion face of a face is specified by 3D location which changes with the time amount of the 
sample point set on a screen. When a face moves and it is controlled by incorporation data, these points are the 
markers on a face by whom tracking is done by the motion incorporation system. The movement toward these 
points is used in order to control the face by the set of a control point which influences the field of the face 
smoothly. Incorporation of the motion data of a face can be performed by one of suitable approaches so that I may 
be understood, if this field becomes a familiarity person. At one example, a motion of a face is work "creation 
(Making Faces) of a face" (Proceedings of SIGGRAPH 1998, pp.55-67, 1998) besides Guenter. It was incorporated 
using the technique indicated. 

[0074] The mesh deformation face is animated according to the linearity-combination of the variation rate of the 
control point set oj by displacing each top-most vertices wi of a triangular mesh from the static position. These 
control points correspond by the sample point pj which has described the motion, and 1 to 1. When the effect each 
control point affects top-most vertices falls with the distance from the corresponding sample point and two or more 
control points influence top-most vertices, those weight is normalized so that total may be set to 1. 



[0076] the case where top-most vertices i are influenced by two or more control points in the above — betai 
=sigmak alphaik it is — otherwise, it is 1. Since such weight is calculated once using the static position of the 
sample point and a face mesh, a motion of the mesh for every frame is sparse-matrix multiplication, (sparse matrix 
multiplication) It has become. The degree type was used about the weighting function. 



[0078] In order to process the details which animate a face, the 2 type exception is prepared in these weighting 
rules. The control point which the tag of a "top" and the "bottom" is attached, for example, the neighboring top- 
most vertices and the neighboring control point of an eye and opening have on opening does not influence a motion 
of the top-most vertices under opening. Moreover, the Scala texture map in the field around an eye (scalar texture 
map) It is used in order to carry out weighting to a motion so that an eye may become the shape of a smooth taper 
and may become zero in the part of a palpebra. In order to move a face mesh according to a sample point set, a 
control point location is calculated and. as it is, the face must be made to deform. If the same weighting function as 
having mentioned above is used, it is calculable what kind of motion the sample point carries out according to a 
control point The result is linear transform p = Aq. Therefore, in order to ask for the sample location pt at time of 
day t control point qt = A-1pt can be used. However, since Matrix A has that conditioning is not enough, in order to 
avoid the field configuration which happens since the motion of a control point is very large and which is not 
desirable, A-1 is calculated using SVD (Singular Value Decomposition-singular value decomposition), and the 
singular value of A-1 is clamped to Limitation M. M = 1.5 is used in the example of a graphic display, as the 
reference which is discussing SVD — Golub and Van Loan work "matrix count (Matrix Computations)" — there are 
the 3rd edition, Johns Hopkins press, and 1996. 

[0079] In order to wear the motion face of an eye and the head the appearance near thing, the motion generated by 
the procedure is added to an eye, and the rigid body motion incorporated independently is added to the whole head. 



[0075] 
[Equation 9] 




[0077] 

[Equation 10] 



h(x) = 14 + VScosOtx;. 
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An eyeball rotates according to the sequence of the random fixed direction, from one eyeball, toward the following 
eyeball, moves smoothly and goes. A palpebra rotates the top-most vertices which define the palpebra as the 
circumference of the shaft passing through the core of an eyeball, and is animated because deformation is made to 
be performed smoothly using the weight defined on a palpebra mesh. 

[0080] The rigid body motion of the head is incorporated from a physical motion of a person's head. This While the 
person is wearing the hat marked with the special machine cognitive possible target, it is carried out by copying the 
motion on a film (a hat). Work besides Marschner "the image base BRDF measurement (Image-based BRDF 
measurements including human skin) containing human beings skin". Rendering Techniques'99 (Proceedings of the 
Eurographics Workshop on Rendering). It is patternized so that what is used by pp.131-144 and June1999 may be 
resembled. By carrying out the tracking of these targets by the video sequence, the rigid body motion of the head is 
calculated, and after it, this is applied in order to carry out the rendering of the head model. Since only a video 
camera is required according to this setup, a motion of the head can be made demonstrating action of a wish. 
[0081] In order to carry out the rendering of the instantiation system for carrying out the modeling of the reflection 
factor, and the face image with an approach real feeling, geometry is not only exact, but it is required that count of 
an echo of the light from the skin should be exact. In the example of a graphic display, in order to carry out the 
rendering of the face, the Monte Carlo beam-of-light marker of the physical base is used. As an example of 
technique, work besides Cook "distribution beam-of-light tracing (Distribution Ray Tracing)". Computer Graphics 
(SIGGRAPH'84 Proceedings), Work besides pp.1 65-1 74. July 1984. and Shirley "the Monte Carlo technique for 
direct-lighting count (Monte Carlo techniques for direct lighting calculations)". There are Transactions on Graphics, 
15(1):1-36, and a thing indicated by 1996. According to this approach, BRDF (bi-directional reflectance distribution 
functions: bidirectional reflectance distribution function) of arbitration can be used, and approximation-ization can 
simulate the difficult appearance of the skin to accuracy with a simple shading model. Furthermore, although the 
extended light source is used, when carrying out the rendering of this like a portrait photograph, it is needed [ this ] 
in order to obtain the image which is not unsightly. He is trying for physical light migration to shift in the following 
two points on the need for count effectiveness. Namely, diffusion interflection (diffuse interreflection) It is ignored, 
and the eye is illuminated so that it may pass without refraction along a pupil. 

[0082] In the example of a graphic display, the reflection factor model of the skin is based on measurement of actual 
human being's face. As an example of technique, there are some which are indicated by work "the image base BRDF 
measurement (Image-based BRDF measurements including human skin) containing human being's skin" Rendering 
Techniques'99 (Proceedings of the Eurographics Workshop on Rendering) besides Marschner, pp.131-144. and June 
1999. These measurement has described the average BRDF of the frame of some photographic subjects, and 
contains the adaptation parameter of the BRDF model indicated by work "the formation of nonlinear approximation 
of a reflection factor function (Non-linear approximation of reflectance functions)" besides Lafortune, Computer 
Graphics (SIGGRAPH'97 Proceedings), pp.1 17-126. and August 1997. Therefore, these measurement serves as a 
starting point which excelled when carrying out the rendering of the face with a real feeling. However, it is necessary 
to reinforce these measurement so that a part of spatial change observed by the actual face may be included. This 
is begun from the adaptation to the measurement BRDF of the skin in which the skin carried out the rendering, and 
a similar photographic subject, and is performed by dividing it into a diffuse reflection component and a specular 
reflection component. After it. in order to modulate each, the texture map is taken in. 

[0083] According to the measurement taken from the actual face of a photographic subject, the texture map of a 
diffuse reflection component, i.e., an "albedo map", (albedo map) modulates a diffuse reflectance so that it may 
explain below. A specular reflection component is the Scala texture map. (scalartexture map) It is modulated, 
specular reflection is removed from the parts (eyebrows, eyelashes, etc.) which must not carry out a rendering at 
the rate of a cutaneous reflex, the specular reflection of the lower part of a face is reduced, and it is made for the 
description of the skin of a face to be resembled. The result is BRDF which changes spatially and this is work "the 
formation of nonlinear approximation of a reflection factor function (Non-linear approximation of reflectance 
functions)" besides Lafortune. Computer Graphics (SIGGRAPH'97 Proceedings), pp.1 17-1 26. and the 
conceptualization cosine lobe of August 1997. (generalized cosine lobes) It is described by each point by the sum. 
[0084] The albedo map carried out the construction graphic display, and the albedo map on which it is required that 
the reflection factor resulting from diffuse reflection which changes spatially should be described was measured with 
the operation gestalt mentioned above using the digital photography sequence of the face photoed under the 
controlled lighting. 

[0085] Drawing 1 1 shows the instantiation system used in order to capture digital photography or an image. In the 
system of a graphic display, a digital camera 1100 is equipped and two or more light sources are included. The 
example of whether to be and to attach is shown by 1 102 and 1 104. Since the polarizing filter which carried out the 
form of the vertical polarizers 1 106, 1 108. and 1110 was equipped and the light source and a camera lens are 
covered, specular reflection is inhibited and he is trying only for a diffuse reflection component to remain in an 
image. In this example, a photographic subject wears the hat 1112 with which the target which can be machine 
recognized was printed, and is made to carry out the tracking of the pause of the head. The camera 1110 remains 
standing it still, while the photographic subject was rotating. The only lighting has been obtained from the light 
sources 1 102 and 1 104 put on the measurement part near a camera. In order to reduce the indirect echo from the 
light which leaked, the black back drop (background curtain) is used. 

[0086] Since it understands, the location of a camera and the light source is a field normal and irradiance. 
(irradiance) The coordinate to which it corresponds in an eye direction and texture space is calculable for every 
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pixel of each image using the standard beam-of-light tracing technique. Ideal iambert (Lambertian) If an echo is 
observed can calculate a Iambert reflection factor from this information about the specific point in texture space 
under an assumption. If this count is repeated for every pixel contained in one photograph and that total is obtained, 
an image will be projected into texture space, and if it ** with the calculated irradiance resulting from the light 
source, the map of the diffuse reflection factor which crosses a field will be obtained. As shown in drawing 12 , two 
photographs (A) and (B) are projected into texture space, and lighting is amended. Actually, this projection is 
reverse mapping repeated so that an outer loop may pass along all the pixels in a texture map. (reverse mapping) In 
order to take the average of the whole image area which is performed and is projected on a specific texture pixel, it 
is a probable super sampling, (stochastic supersampling) It is used. 

[0087] The albedo map from a single photograph covers a part of field, and the result is grading, (grazing) It is best 
when an include angle is small. Therefore, in order to create the single albedo map of the whole face, the weighted 
average of all individual maps is calculated. Although the visual example is shown in drawing 13 (A) and (B), a 
weighting function is chosen so that high weight may be given to the pixel displayed and/or irradiated from the 
direction which is making the normal mostly to the field, and before either a display or an exposure carries out 
grading to the degree of pole, it descends to zero. The following function is used in the example of a graphic display, 
(costhetai costhetae-c) p however c= 0.2, p = 4. 

[0088] A pixel is visible and irradiating suitably is confirmed, before calculating the albedo of a specific texture pixel. 
Even the camera point is traced with the point to the point on the light source on the pixel, and, as for two or more 
beams of light, a mark partial [ visibility and lighting ] in ** at zero or perfect is attached to the pixel. When 
estimating the size of the light source, it is wise to avoid a serious error. Only the albedo of the pixel which it is 
completely visible, is thoroughly irradiated by at least one light source, and is selectively irradiated by no light 
source is calculated. If it does in this way, the pixel in the pixel and all the shadows, or penumbra field hidden 
selectively will not be used. 

[0089] Some calibration is required in order to give semantics to these measurement. The imprint curve of a camera 
is proofread using the approach indicated by work "recovery (Recovering high dynamic range radiance maps from 
photographs) of the high dynamic range radiance map from a photograph" besides Debevec, Computer Graphics 
(SIGGRAPH'97 Proceedings), pp.369-378. and August 1997. The flat field response of light / camera system is 
proofread using the photograph of the big White card. The focal length of a lens and a strain are proofread using 
Zhang work "the flexible new hand method for a camera calibration (A flexible new technique for camera 
calibration)", TechnicalReport MSR-TR -98-71, Microsoft Research, and the technique indicated by 1998. The 
whole scale factor is set using the reference sample of a known reflection factor. When change between the images 
of light source reinforcement is set as the object of consideration, control is performed by including a reference 
sample in all images. 

[0090] The texture map obtained from the above-mentioned process is carrying out good work, when incorporating 
the detailed color change covering the whole face automatically. Furthermore, the strap used in order to hold a 
calibration hat in an orientation is visible. An image edit tool is used for these problems, and they are solved by 
embedding a texture or a homogeneity color in blank area soon. 

[0091] Drawing 14 (A), (B), and drawing 15 (A) and (B) show the albedo map before edit, and the albedo map after 
edit for the purpose of a comparison. It is clear that be [ the part where an appropriate result is not obtained from 
an albedo map / where ] it is the part (for example, under a jaw) where a field is not fully observed, or is the part 
(for example, lug) which scans correctly and cannot be recorded with an image since the field is making the 
complicated configuration remainder. Neither of these parts need the texture from an albedo map, in order to 
acquire an appearance with a real feeling. It is because the 1st part is not visually conspicuous, and the 2nd part is 
because geometry is a detail in visual. Therefore, even if it edits these parts, most appearances of a final rendering 
are not influenced. 

[0092] DrawrngJ 6. - drawing 18 are drawings showing a different modality of some of face models which used the 
rest frame from the accompanying video. The face from some include angles is shown in the 1st ((A), (B). (C)) line. 
[ of drawing 1 6 ] There, it is shown how an albedo map and measured BRDF incorporate the appearance and color 
change of the skin covering the whole face seen from the include angle of arbitration. The 2nd ((A), (B), (O) line 
shows the effectiveness of the lighting of a rim and side including the strong specular reflection of whenever 
[ grazing angle ]. [ of drawing 17 ] In addition, all of three images of this line are the same reinforcement, and the 
light source is put on the same distance from the face. Direction change of a reflection factor is contributing to a 
light effect with familiarity showing up in a rendering. By the 3rd ((A), (B), (C)) line, even if deformation of expression 
is added to a face, it is shown that a face seems to be natural in a motion of the usual expression. [ of drawing 
1818 ] 

[0093] Drawing 19 is a flow chart which has indicated the step of the approach for creating an albedo map according 
to the operation gestalt currently explained here. This approach can be realized in suitable hardware, software, 
firmware, or such combination. With the operation gestalt currently explained here, this approach is realized by 
software as what is used by the same system as the system explained with reference to drawing 1 1 . 
[0094] One or the two polarization light sources or more which can be used at step 1 700 since a photographic 
subject is irradiated are prepared. The example of the light source is as having mentioned above. With the operation 
gestalt currently explained here, the light source is chosen so that the specular reflection component of the face 
reflection factor of a photographic subject may be inhibited or removed. At step 1 702, the face of a photographic 
subject is irradiated according to the light source. At step 1 704, a photographic subject is rotated and a series of 
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digital photography or images are captured by the meantime. At step 1 706. the coordinate in a field normal, 
irradiance. an eye direction, and texture space is calculated for every pixel in a texture map. This count is performed 
using a well-known algorithm. At step 1708. the lambert reflection factor to the specific pixel in the texture space of 
an image is calculated. Thereby, the albedo of a pixel is obtained. At step 1712. it is judged whether the additional 
pixel remains in the albedo map. If it remains, the following pixel will be obtained at step 1712 and it will return to 
step 1708. If the additional pixel does not remain in an albedo map, it is confirmed at step 1714 whether the 
additional digital image remains. If the additional digital image remains, the following digital image will be obtained at 
step 1716. and it will return to step 1706. If the additional digital image does not remain, at step 1718. the weighted 
average of the individual albedo map of each image is calculated, and the single albedo map of the whole face is 
created. The example of how this weighted average processing is performed is indicated by Marschner work "the 
reverse rendering (Inverse Rendering for Computer Graphics) of computer graphics", PhD thesis, Cornell University, 
and August 1998. 

[0095] Drawing 20 is a flow chart which has indicated the step of the approach of calculating the albedo of a single 
pixel. This approach can be realized in suitable hardware, software, firmware, or such combination. This approach is 
realized by software with the operation gestalt currently explained here. At step 1800, when a certain pixel is given, 
it is judged whether the pixel is completely visible. The albedo of a pixel will not be calculated if a pixel is not 
completely visible (step 1804). If the pixel is completely visible, it will be judged at step 1802 whether the pixel is 
thoroughly irradiated by at least one light source. The albedo of a pixel is not calculated if the pixel is not thoroughly 
irradiated by at least one light source (step 1804). If the pixel is thoroughly irradiated by at least one light source, it 
will be judged at step 1806 whether the pixel is selectively irradiated by one of the light sources. If that is right, an 
albedo will not be calculated about the pixel. If the pixel is not selectively irradiated by one of the light sources, the 
albedo and weight of the pixel are calculated at step 1808. This weight packs an individual map later, and when 
taking that average, it is used. Therefore, as mentioned above, it is completely visible that an albedo is calculated 
and it is only about the pixel which is irradiated thoroughly and is not selectively irradiated according to one of the 
light sources by at least one light source. If it does in this way, the pixel in the pixel, all the shadows, or penumbra 
field hidden selectively will not be used. 

[0096] As mentioned above, based on explanation of this description, if the application program for realizing the 
approach for the approach of animating the description of the explained face and expression conversion is this 
contractor, if it is this contractor, it can be created. 

[0097] What is necessary is just to create an application program with the instruction which can perform CPU and in 
which read activation is possible. Moreover, one or two or more are [ that what is necessary is just to memorize the 
record medium which memorizes the above-mentioned application program to memory, a hard disk, and a well- 
known portable record medium ] sufficient as these record media. For example, the data used with application, for 
example, the data which defined a code book and training expression, are memorized to CDROM, and the application 
program which uses these data may store an application program and data in a record medium which is different as 
it memorizes to the hard disk, moreover, the thing which may be made to record on the same record medium — it is 
natural. 

[0098] The computer which furthermore executes an application program may carry out distributed processing of 
the processing which one set is sufficient as and was specified with the application program by two or more 
computers. 
[0099] 

[Effect of the Invention] After meeting the high criteria that carry out the modeling of the face to a face with a real 
feeling, and it carries out a rendering to it according to the operation gestalt which has carry out conclusion 
the system and approach of solving the difficult problem of enabling it to display in the form where the image which 
has familiarity like human being face is reliable are offer. The view of this approach is to use measurement as much 
as possible so that a face model may resemble an actual face actually. The geometry of a face is expressed in the 
fragmentation side with the connectability unified in common with a different face, and response nature mapped with 
the variation rate. It is obtained from BRDF measurement of the skin of human being by the former, and the 
measurement is used together with new measurement, and the reflection factor is combined so that two or more 
views (view) may become a single lighting amendment texture map showing a diffuse reflectance. A motion is 
obtained by the technique which was mentioned above and which is moved and incorporated, and the motion 
suitable for a shading side is made to be obtained by applying the motion to a face model using the improved 
deformation approach. The real feeling of a rendering is the renderer of the physical base about the geometry of an 
actual face, a motion, and a reflection factor, (physically-based renderer) It is improving substantially by using it. 
[0100] As mentioned above, although it caught from the structural feature and/or the approach step and this 
invention has been explained, this invention clarified by the claim is not necessarily limited to the concrete 
description or approach step mentioned above so that I may be understood from the above explanation. Rather, the 
concrete description and an approach step are indicated as a desirable operation gestalt which realizes invention of 
a publication to a claim. 
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DESCRIPTION OF DRAWINGS 



[Brief Description of the Drawings] 

[Drawing 1] It is the block diagram showing the general purpose computer suitable for using it when realizing the 
operation gestalt of this invention. 

[Drawing 2] It is the block diagram showing the system which can be used in order to incorporate simultaneously the 

structure information and reflection factor information of a face on a photographic subject 

[Drawing 3] It is the flow chart which has indicated the instantiation approach for incorporating structure 

information and reflection factor information according to the operation gestalt of this invention. 

[5 r 3M0.g_4l It is the explanatory view showing the instantiation code book and transform function by the operation 

gestalt of this invention. 

[Drawing 5] It is the flow chart which shows the expression translation process by the operation gestalt of this 
invention. 

[Drawing 6] It is the block diagram showing the instantiation system which can adopt the principle of the operation 
gestalt of this invention of a certain kind. 

[DjrawiQgJLl (A) - (C) is drawing showing the collection of the instantiation collar plate in which the instantiation 
expression conversion by the operation gestalt of this invention is shown. 

[Drawing 8] It is drawing showing the process which maps the same fragmentation control mesh in the fragmentation 
side by which the variation rate was carried out whenever faces differed. 

[Drawing 9] (A) And (B) is drawing showing the instantiation constraint used in order to match the description in a 
field adaptation process. 

[Drawing 10] It is the flow chart which has indicated the step of the field adaptation approach by the operation 
gestalt of this invention. 

[Drawing 11] It is the block diagram showing an instantiation system employable in order to make the albedo map of 
a face according to the operation gestalt of this invention. 

[Dr awin g 12] (A) And (B) is drawing showing the instantiation albedo map of two photographs with which it was 
projected into texture space and lighting was amended. 

[Drawing 1 3] (A) And (B) is drawing showing the instantiation weighting function corresponding to the photograph 
shown in drawjng _1 2 . 

[Drawing 14] (A) And (B) is drawing showing two perfect albedo maps to two different data sets. 
[Drawing 15] (A) And (B) is drawing showing the albedo map of drawing 14 after editing. 

[Drawing 1 6] (A) - (C) is drawing showing the collection of a face model by which the rendering was carried out to 
different sense under different lighting conditions. 

[ Drawing 1 7 ] (A) - (C) is drawing showing the collection of a face model by which the rendering was carried out to 
different sense under different lighting conditions. 

[Drawing 1 8] (A) - (C) is drawing showing the collection of a face model by which the rendering was carried out to 
different sense under different lighting conditions. 

[Drawing 1 9] It is the flow chart which has indicated the step of the approach for creating an albedo map according 
to the operation gestalt of this invention. 

[Drawing 20] It is the flow chart which has indicated the step of the approach for calculating the albedo of a single 
pixel according to the operation gestalt of this invention. 
[Description of Notations] 
200 Lighting System 
202 Camera 

202a The lens of a camera 

204 206 Light source 

400 Code Book 

402 Training Expression Set 

406 Conversion Processor 

408 Transform Function 

410 Synthetic Expression Set 

602 Transmitter (Transmitting Side) 

604 Receiver (Receiving Side) 

606 Reconstruction Module 
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- K 1 6 6 -*»#>f yf >-^t^M ^168 te¥<DAti 

-f) £ LT^Ttgft t>tf>£ UT, -e-f*n*>\ v? g * 

X^S^, >f—K/< y K, t77-fh7>f5/*>a> * * 
70 tt, 1 O^fcll2O^_bC0^7< 7ft ±15*5 

.fcUfft^Art^-f *fi. 1 3 6 CUB'S' £ ft fc'f ^ 
* 1 7 0 SriiLXteJl^^y h 1 3 2lcgMK£ 

^tT^So 1 7 2 *fcflte<*>* 4 /<D7^ 

-fx'^ t'7t7mi7 4/^^'fy^7x 
7Wc, yt;^ > t°zL — ^fi, ^ tr— ^-^^y 

[0020] ^yb'a-^i 30it y ^e— h ^vfc'a 
^ — ^ l 7 6 J: 9 ft, l o^/cli 2oJ^±o y ^e— h ^ 

a-^l 7 6^:1X11 S'J^^^°— yf/ua >- t°3L — 

[3 1 7, h l/-^r/M ^178 /cit^T^^fL 

aye»-^ 1 3 O^gljgLTiiiliLfci^U 

^>5 0 [§] 1 (w^i-f^S=i^^ ><b UT*1, n-^/U 
xy/^7 h!7 — ^ (local area network - LAN) 1 8 
0 kfct$L^V — 9 (wide area network - WAN) 1 
8 2^£>5 0 r <DM(D*y V 7 - i?m$tte. *7 

K fcit/-Y>^-^y h (the Internet) "CH^ »9 -S- 
^fc?*S£irftor^S 0 
[0 0 2 1 ] LAN* y h >-^^JtX<£ffi$^6 t 

#11, ayt'a-^ 1 30(i, — ^>fV^7x 

-^^fcflT^r^ 1 8 4 ^rilLTta-^/U^-/ h 57 — 
M8 OtcS^^ttTl^o WAN* y h7-*^?mm 
40 X'tizmZtlZk #11, ayfa-^l 3011 >f 
*?/ h<7)J: 9 ft, fi«*y h!7-^18 2i^)a ^ 

V^rSii:-r5^^l-, ^-IrA l 8 6 £fcll* > 

^^>«9, ^^«bt> 7fr«— h-r y^7x-^ 1 5 

6 ^Lt^ 1 3 6 JC»a*ixri^o *JX h !7-^r 

>^/^ifr*fi, A*-yt/ua v t'zi.-^ 1 3 ofcgiigL 

X±&Ltzy'v ?7 J**i?~-/l'£tcn*:<D— sbji, y 
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[0 0 2 2] — ftttfjlC nyt'a-^ 1 3 0 
^^v/^fA|t WJLlis 7d 7 t'f^f^^ -^CD-ROM 

ou^&3 vt° *y ic-f h— /u&fzn 

[0 0 2 3] jSHWftSl, \y — "T^ >y>*^J± 

^ hi, ^i^nar^yfa-^ogi 

[0024] mm (ommt \htst^<pm^^ 

& 9 7^ (capturing system) (2. 

(structured light) LtS^lfia^^ 9 i£tf 

(reflectance) j^tf>£H4£:& 9 iiA/ti^j: 

irirLTte. t a !l f y t ^a-f Pfl©^ 9 
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[0 0 2 5] «^(*:<0«^«»ai:Rlt*»tt 

U CCDT u^f CCDT u--f Z S? 

(facial illumination system) t) i£{fi L "C V ^ 

(genlock) $ixTl^5 0 HI ?n (£> ^ T 1 f 1 , 20(D 
M2 0 4, 2 0 6^?lJffl$nr^€>o *!2 0 4lt 

Zs (structured pattern) %\tit}-tZ> 0 M2 0 4lis 
20 aSftn^r-i/a >T**>ntf, ^ ^ T* fc B < - t # ^ 

* L < ««>RI***Mt«)»«Rltrt5>S:* p< 7 2 0 
2T-®9&*!>6 «t ^ (c-r63fe^m^-T6o *I2 0 6 
fis 0H;tf*, t'-A^^y y h (beam splitting) 
7<OU^2 0 2 a t f^^J(CBl/>T *3 < ^ 

JHfix SW^^^iSMS^ (specular component) ^ 
X*n. t&MMyt? ^ (linear polarizing filter) 

m&i»Mmft ? ^ ^9 twis^-r 5 ffii # iw & o r ^ r . u 

VX2 0 2 a C0Bff|rB^ttX^5/c*!>. m^h^m^K 
*t-D-x<>K (W^L^. 780-880 nm) |C -6 X 0 Iwil^ 

•rst, ^iMsw^tsc^^T'^o CUB** 
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[0 0 2 6] «»0«*63ftW&ttflJ-r*4:* 
SKI******:!*^-* (3tjR2 0 6*>5><0) ti 

f^-Cffiffl-C* SWftSfcfi^--* J&s»<b*L* C £ left 

6 0 WT-eiftM-t-sw-ett, rcot&sitts^Tcx-^ (3 

Dt^— *) fcT/^Ktes^^ (albedo map) ^§A/ti> 

[0027] 1311 :ccMt5^M8i:J:5* 

&>Z>o ^ry7 P 3 0 2T\ X X y X 3 0 0 IDfBMfr b \s 

y^yyf-^ ( J f^^^y^ J f—f) tmi&f-* 

tm&'T — — ^ 2 0 8 (H 2) (w 

4 T\ ftLfflgSx ^Xu — h (generic face template) £ 

i^y^^f- ^(C3iffl-r5<h, L^y^yT'f-^ 

[0 0 2 8] ^fi/7 ( 3 0 6 t% U>-v^ yff— fit 

«rt6T-^>i9. ztm. z<Dftm<omm#\z&w<D i><ox 

T^&^Ot^&So i-frfr^. Turk & LevoyS fuyy 
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B4^5><Oi/yA' Ktf? y 5/ (Zippered Polygo 

n Meshes) J (SIGGRAPH94K }3£TfF. Bernardini, J. M 
ittleraan. H. Rushmeier. C. Silva. G. Taubin^fc^f 

(The Bal 1-Pivoting Algorithm)! (Trans. Vis. Comp. 
Graph. 5:4 (1999)) XhZ> 0 ^T^3 0 8t\ 
;yX3 0 6 <D3DffitC^-r-5S&^^<^ h ( 

«fe*fijffli"6^ t t^Tlfi-CfcSo &f-> ^f^3 10 
•C, 0 8 tttl^tlfciifei^^f 0 

X-Mi, Marschner^ raytfa- # ? *7 7 <i y V *<F> 
igfUV^y >^ (Inverse Rendering for Computer Grap 
hies) J (PhD thesis, Cornel University, August 199 

8) fcEttSttn^o 

[0029] :^fi-e, ia 2 t^-r ^ 5 1-. mw&m 

[0 0 3 0] ilEBaiB^aStt. E5*Sttr^-6««*tt 

[0 0 3 1 ] a— K^y^^ffiffllcJ:6*«K* 
*>-5a»**H2£»ffi-Ctt, a-K^^ (code book) 

4OO(D0i^^t o ClfrtbtO^ff H ^Iffl^ff? (generi 
c expressions). "9 , 4#^(7)fi A*^6 "Ctt* < > VI 

Witt. Mz-tf. r^-^^yuj -cfco. ^ff2tt rt^ 

£>®J (CT-fcO, *«3tt r^g r> j (Oilgffe^o ^ 
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[0 0 3 2] SWXftSrfir 3-h7^4 0 

Ol^Sffftf) — Sn^ffiffiL-C, ^&B§^ (transformati 
on function) as»«Jfci»# Hi S*b*. 
ttJ-r/b^lC h \s — (training set of 
expressions) 4 0 2 5 % jLJj&&*) J^tlbtlX^ 

C^lTl^o Wttf, M/- = y^4 0 2li, 

?ti r^^^/uj % ru*»*Bj *sJ:W rjgoj icfco 
hy * K^y^ 4 o o (D^it^-r-<x 

^yy >y-tz> e w-c*giJA«^K«-*««wiafflstts 

v^j* h y , Ki«oPfl<oJ:5i:IS 

[0 0 3 3] B4l:Sot»Wt5i, 

^i/ot^f 4 0 6^$ttTt>6o ZfD&t&^wt 
jxt4 0 6fl, ISft^- K*~7\ y^h^^T, 7 

4 0 6 h !>x7X*HSS^tt^ 0 f&MUi&y* 

t!/tiin/-=^^s***4 0 2 i, 

K^y^Sft4 0 4 t tAAt Lt$»t»5. ^^^d 
-fe^*ttr*x?jOA*Srft!'SL-C. ^1^4 0 8^ 
^ttito r<0«»BI»4 0 8 a-K^yMOOfl 
^a«<Oi-^<r^igffli--6 ^/&^ff 31^ (synthesi 
zed set of expressions) i£1&bti,2> 9 Z (D&f&mffiM 

[0034] sw^ra^-cfc-sirsoKoasfad. 

loll, 0 ii^i^V^/UTK^ v h CoAfflt* 
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jESSIc— ai4i^5)tfc5. tilott, m<nM 

4 X«l Aft £ A4&<0ffflT-aHb-r 6 b "C*> *• 
[0 0 3 5] ilO^-ail, -T-<T<Dffi^$!j#&t>& 

70 "CJi. — JtffiS**!* F^^1r>://MftB(standardsara 
pie positions) J ^If^tl^o SU* ^ U V ^B8» 
fl^ 5/ '>3LS^§i^C(mesh deformation function) T*&> 

[o o 3 6] i2(o^F- ffc-cti, *>6«^e>sij<oM— ae 

^OOSJ# iCOV>X«. 1 0, 0 0 07l/-ix(t 
ft (time warping) g Sb^d T 7 -f * ^ h 

[0 0 3 7] ^flfli, mflOS2P^^^Wifi"C^x, 
y, z^f4^-T-<r ^r^A/X*l^3m^<^ h /Ugl- <£ o 

bl...n iHfl^SJ^ (linear predictors) M 

[0 0 3 8] 
II&3] 

[0 0 3 9] g|^^)^T-|L ^&g a j^^^ > htt/h*ft 

(average valence) li^j6X**> 5 <^"C> ft&tflttgajttl 
8S$lC^ot^5o ^(DjjgmcOi/^x^^Afrn x 18 

[0040] msk<omj&isx7M>n&ifrttrt&+&xte 

ifefb*iX*v\ 4^^fiS (singular value) GO&lffcte, ^ 
fcliS?^^ (pseudoinverse) ^rtt^LT. ajO^^jft^ 

so a i5fcffi<D&g i mn'r<<xi? pi^n^o fcfcu <mi* 
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h y y*X<Dm*:&g<mX~tb*) s a = 0.2. ...0. IT 

[0041] men, ::r^it^6SMi{:J: 
hx£>6 0 ^75//5oo-e, ^tt^=i- K^y^^ffl 

^5/ **«#»e>*ifcA»ifiS45Atea>fc^*ra^ 
LX&S&frr <h#X£6 0 h ft 

a. — ^ (neural network) 4if<0* 

^SMMttfflt5^i:t), (basis expr 

essions) <0*nfifq r £ fc WfgX£>5 0 

[0 0 4 2 ] W^cDT^y ^r — >3 >- 

^&;/nir*& 5 C^D AtiZfrfiK \nnmx 
?f£*lX^<5o ->^f^6 0 OliSfliJ^yt^-r^ 
y^'>^f^, 0£9, h7^V7^ 6 0 2 ^ §<t 
yt'a-f^ isPisa^J** O^D. U 6 0 

4h&^fr, ZtlbU. << h (the Internet) 

t£Z<T>* V YV-Z 6 0 3 £^ LXi§f§"1~5 J: 9 dig 
^^ttr^6 0 h7y^vy^ 6 0 2 fifiHM^^-r^ 2 
00 (132) rtttt. (3 2^r#^LT±a!Lfc 

X o fe^A^^^W^^^^tfct 5 CM^tv^ 
5o SfcK* h7>^V7^6G2|j: x H4Sr#fiBLr 
JLSBLfc. t^<b[s]i:J: ^F7^ 4 0 0^ 

RSP^v-^xA 2 0 0#&9 iiA/fc'aWSrfcoA*^ 

[0 0 4 3] U> — /<6 0 4 IIWS^^v^zl— yP6 0 6 

h7>^ v^^ 6 0 2^p>SflLfcf 

tiC^ - K^7^ 4 0 0$:^A/T*^6o ^'XIS, h 
7 ;7-^ 6 0 2 $Jl;il/>5A^ri 5 , U->— v<6 0 4 00 
(-t>5A$3£ai{f L «fc 9 <t LXi>£ 0 h7V^;^6 
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mmtmzn V 7 $^^6 0 2l:J:ot»>)iit 

0 6 LX^filS^tL6o 
10 [0 0 4 4] Wm<DMWSk 

mm. r zxfow^x\,^z>mfaMmz£z>min&&<D 

IlA^ (A«A) -€rr»w« 
3o<7)S^6^ft^^^ttX^-5o m^lfete, Afe 
Ac^^O &*ftfc«<Z>»>#tf>|g*-e*>5. S2A^5 (A 
«B) <B«0>»#fS. *rX(-Si!9i^^nx^-5o A^BO 
WlV &i£titcm<nW)$te3?tB (C) tc^^^tx^5 0 

A»<0«<O±KJR#. A<ft^«*saj»Sttfct# Ks/ h 
^ 0»#^aiJ^i-^w^(cJ:oXSit)^^nx^5^. r 

^JzoXBiDi^tpr. <^^x*#6o A^BO^Oi^^n^c 
tb^tix ±a>UfcJ: 5 twffiffl^tbX, A4*A(&*tiias3£ 

i^tltl^o 2ff ilw^^ttXV> 

5 0 ^tt^> 3ir h^i^^^tf -T^XtSSiw-ifeU 
XV>-5 0 flJ)[g]{^-rJ: 0 1^. lf?@ (A) £2?t@ 
(B) (OS«li, *«l:R4 6 2 A^<b*t£UJ £*x*i 
^LXt>. **^«ffilUT^5^)lC*tL, S2A^ (3 
30 fTS (C) ) <7)^y ^-f/U^iffi, lffi (A) ^2t? 

B (B) (Dmffih&&ftftlzMUl*X^te\,\ 

[0 0 4 5] m^^^&fflWk^&tcfr^ffl^i/XTJ^t 

rtu «<o^^-^ hy (o^t), ciiit 

t)> iS<0*7-^Stt«Fl4) Ci^lt^5 0 r coir ^ 

[0 0 4 6] mm?** hy«. ^^ffii^, s^s-rs'i 

(DS^^D^Ac^^^^fil^^ttX^^o wOO^XIL & 
^#ffi^^t, iy7&t>otoftffi'Cm£tiX^Z 9 

[0 0 4 7 ] * y > z*. ig-& 
5^7 *ttt?iBllSnfcv;t^ h y lw®mKi£(S*-r£*8#® 
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op<£>$3;#/W — /Ml, Charles Loop^ r=ft^^S<5<¥ 
HI SB #35 (Smooth Subdivision SurfacesBased on Trian 
gles)j (PhD thesis, University of Utah, August 198 

7) ic»u<KMSixr^-5o £h\^. mftmn. Ho PP e 

te^ rjK^^filWfttl^Piecewise smooth surface re 
construction) J (Computer Graphics (SIGGRAPH '94 Pr 
oceedings) pp. 295-302, July 1994) dfEft $ thX l> 

5 Q ^F^J^^y— X-^*^ (crease mask) llffi 

fflSttT^*l\ S^K. ^-y(dart) <t^!J-X]H 
(crease vertex) COF^CO^ -/ is&tmjfrik'lr 2> t # . ^ 

y - xm jR k * ^ £ »t # v^-^ * ^ t 

[o o 4 8 ] -r^xom^'rM^n't&mft^te. *s> 

LTJfcftStu XgjJMfcBfcH- 

6o ^^(Otai'Cfi, 2 2 7 m<DTRM>k 4 1 6 
5****>9* ijxfcte. P<oBBS^M«J«w=x-^S:f^j58; 

v\, miCTl^ (isolated vertex) fcttfis ^5tf-£^ 
£;ft,fcPB#i£J^ (limit normal) t>o * t ft ^ 
ft* 

[0 0 4 9] 08fi, Z<Dffl{Z'&oX®im£tltc®.7£m 
[0 0 5 0] »a»SS:«-M^ig^$-fr-5fc:^^ffiffl$tt 

5^Dir^fl x Hopped TE5>lp^®^P^(Piecewis 
e smooth surface reconstruct ion) J (Computer Graphi 
cs(SIGGRAPH '94 Proceedings) pp. 295-302, July 199 
4) \Z IS1K £*LT^ 5 T/U=f}) X^lrS<5t>T^5o Hopp 

hnkWo zb&x% 5. i-ft*>*>. h^p^llf 
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[0051] 7x-xi xn. 5£&<Dffi±iktzntfm{z 

fl, > ^60^^14 (connectivit 

y), &£rj*:<D&mfrtbZ>o 7x-X3H, :7a,-X2 

■tz>^*;i'*mm%&mik-rz>z t\zx<>x. mmx. 
ffimt£m.ft¥-mmftm*m'£t£'&z>o 7x-x3t-M 
fb^fTfc>n-5 mm* > is~ft<Dis&<D» > ^0>mn 

^ lel^fi. r<D7x- X^)^pj^/iS5)^/j:oti^o 

[0 0 5 2] rcoHJS^^X'li, ^^CDf±*^ x Hoppe 

tps. mm* y v-^o^jSttSr^iE-rsi^s 

[0 0 5 3] a^OWttt, ig'&^n-t^^^cDiflgia 

^S/Hb-T6o 

[0 0 5 4] 
^ [IS4] 

E(y) = £j (v,p) + A&(v) + /i£c(v) 

[0 0 5 5] Jifat-^^T, vte-f^X<Dm&&m<D'< 

^r-T-^sK-f V h*>P>*l^ffi*t?COft— *8EBt (sum-squa 
red distance) %Mfe~f~Z>o 
[0 0 5 6] 
[ife5] 

40 &(v,p) = Ea^-n(v, P or 

1=1 

[0 0 5 7 ] ±!5iCiol>T. Pi»iI#@C0U^v?7K>< 

nu, Pi ) m&{iLmvx-mm£titzmftmjL<D 
^rtt(oa«fa)^n(v, Pi ) (c*5ft6ffiifejBi:-au 

5^ [0 0 5 8] 
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me] 




if{s<J>i), n(n(v,/;0)> 
otherwise 



0 a* </ ||/w - FI( v, < rfo 



[0 0 5 9] Jiiaic^o^r. s (p) «\ > hptciott 

[0 0 6 0] ¥miH^E s te. mm* jy>»«rn-*yufc 

[0 0 6 1 ] 
[£fr7] 

I dcgfo) 



&(v) = £ 
>1 



vy — X 

deg(v>) ,ti 



Vh 



[0 0 6 2] Ig,Sv ki |i, vjOP^]S,^T-fo6o 

[oo6 3j mmmm c n. mmzn. mmmM.m&fa 



bttfo-t 5> * - V v h<£ 

[0 0 6 4] 
[&8] 



3: "C-fiOft ~^8§g£-C*fo 



£c(v) = £[,4«v-<A| 



f-l 

[0 0 6 5] ±IElC^o^T, Ajii, j#@OTl,^cDPS^ 

lcfrfc>;ft,S w t*«*JW ITt^o »J*Jtt* Sim* 
nt-lfe* LTV^5 0 (219^. 9 0 0-C<Dm&m'liQ* y is 

^ (a) <D®mt. m**rfr (b) ±o*t«?-ra#>f ^ 

[0 0 6 6] E(v) SrfiyMBfc-r-Sr. <tfl„ n<hai^v(^ 

L, ai £-£{;if£^ n(v, Pi ) Sr-^^jB^ttSiJ»ffi 

srfr-r* sZtte, &m<oi$ky> mLx?tt>ti oo* 
si) , mmn. iiu^iiff tm^. m^tn * >r *s> ~ 



JO 



20 



30 



40 



50 



n&m*v*. n(v tPi )<Dttwmm\z£<>x»:*z> 0 

[0 0 6 7 ] ^tr<ntz#><D* y is tz#> 

z>z k\z±'Dxnm£tiz>o zti\z^>^xn. leemm 

t%ittLi(®ftW (Displaced Subdivision Surf aces) J (SIG 
GRAPH '00 Proceedings, July 2000) (Ctfc^ £ tlX l/^£ 0 

(built-in set of mult iresolut ion basis functions) 
[0 0 6 8] 010(1, ZZXWtW LTl/^H^T^ffitw 

&^xm*T'i'*mm^ztzft<nfim<DXTy7r$:&m 

[0 0 6 9] 775/^1 0 0 0^ 1 o^fcH2oieJl± 

(o®(03D^-^^fflij^$n, ^jc-rea^^/^^fcix 

5o ±&<DMX&^ 3Dy*-?n. S!:U-f uy^t^ 
5 0 ^7-^1 0 0 2T\ l o£fctt 2 o£Jl±co^^e-r 

«I!LAc0ilTMi, ^IffliS^eT'/HL UE.ft*yi/=x. (coar 
se triangle mesh) <DMfc& \-,tc* y ^- t?|3e(C /«f o T 

yv/^-H, ®or>^^y h U K&igClifi 
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(^tT^>ix6o ^f^yiooe-c, iRJB®^-7vw4 i o 

ttS^-C, ^ y ^~<&igfiH4as3E3E£*t6r £»4ftu\, 

se>iw, c^ii^^Dir^xM4, m*<D#"( > b&tite 

(smoothing terra) i6Si6J0SiX. r*U4, fH«M 3/ 
[0 0 7 0] @Oii^JD 

f4BfcBB^*»Wftra«*Bi0&£fti^-e. / y>^ 
f4, 8U*fc^/Mb3ixfca*iBin+*:: ^CJ:ot, 

*i,fti^£>-C\ y 7/u^|(D^i^^t5 £l4RBbftv^ 

[0 0 7 1 ] I^O^g^f/UT^^fl ^^1^7/^ 

14, 2»#rt>6ft**^*«»i-*fc«>{;i % mascot 

co^e^/ur^t). m2U#nn\sii9 <nfcism<D*ry*/i>x~ 

3ft5ff**fCl^* Cl<Df52g&#f4 regg5®(orbit surfa 
ce)j tv*J£^ t\z-TZ>) 0 SSr^flcffi^-r/KO-SB^ 

ot^^wftitt)^ RtBBSdSfift 9-&ofefiBf-e 
^l:g^5J:^l:t^ 2 oUi4^T 0>H<CMfc 

R®T^ jy h£iv 1 oo> * v^Kft* J: Sl-I*^ 

[0 0 7 21 ±&LT#fc3liS^ffi<0fi|jft<B*>'5 

&tz.te*7 tmhti&z. txhz>o mmm* 
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\zis^xh. fo^^^o^icfc^^^coiK^ v Nettie 

<D^±<7)pcofePco8b#^iiJ^$ixfc^-r6o ®A— <D 

z>±x**ixxhz> 0 

[0073] m<pm% 
20 &iz&it'tz>2mm\z&oxm7££tiz>. m&ni&v 

&&T-*lzi:vxmfflistiZ>bZ, ztib(D7?^ v h 
(4, H^aD^^f Al:J:oth9y^^$n 

##4. mffi<Dmmz*j*-xizmig'rz>mw#'< > h<o 
M&izxvxmm&mw-rzitMzifcmztiZo m<n® 

tiZXolz. ^i"tt^(oaaft*&-CfT5 w k&XZ 

h 0 1 OCO MrfcmX 14. ffi^»b#f4. GuenteHtfe^ 
<Df£/£ (Making Faces) J (Proceedings of SIGGRAPH 199 
30 8, pp. 55-67. 1998) KfE® £ ttT l> 5 ^fe^r^ffl L T 
Si?) i££tLTWc: 0 
[0 0 7 4 ] > 

M^l f»lSlT^6. #©J^JtK^ V h^TS^tcS^ 

40 ^(w(4, ^ix5>^>a^l4tt?P*s 1 lift-5<t 5 «wiEa<k* 

[0 0 7 5] 
[^9] 

[0 0 7 6 ] ±IBiC4bH>r, m&i&W$k<offlW#'< > 
hl:J:otlSJii6^ =£ k a ik T**> 0 . * 

50 y v km* v->=.<D»±iiLm$:tem Lx-mft&znz 
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y $ ^HIH (sparse matrix multiplication) (C/^o"C 

[0 0 7 7 ] 
[glO] 

Aft) = 14 + VScosC^x;. 
[oo78] ®&r~>— >a >{k-r6SQ^$:^^-r^ 

f 7 ^^ (scalar texture map) — X 

X¥<D£ 5*tb#^-re^*StSE-T5r k&X$Z> 9 * 
^^ffi«pt**«>6lwrt. SS#P#>f >- hq t = A^pt^r 

^"f-S fc#>IC N SVD (Singular Value Decomposition — 

$^Tt>5o SVD*a&CT^6##:fcfR& LTrtU Golub 
^JltfVan Loan^ fv^ 5/ ^ ff^ (Matrix Computat 
ions) J i£3)SL Johns Hopkins press, 1996#>#> £ c 

[0 0 7 9] @ £§I<t}©j# 

fib, ^«yr>^l(:«i^fcl^ffllt^. 

[0080] m<nwmmW)nA&<n^<n&^#ityi%frh 

*->fy ^Zmm LT^^P^Iw^rO 

Marschnerftfeig r APa1<7>j£j8 £^atf U^^ — XBRDF^J^ 

(Image-based BRDF measurements including human ski 

n) J * Rendering Techniques '99 (Proceedings of the 50 
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Eurographics Workshop on Rendering), pp. 131-144, 
June 1999-C<£ffl£*xT^£ i><DiZ&m-fZ>± 0 
— >it£tlXl^Z>) o Ztlb<0*—# y h £ \?7**i>"- 

tt-g£*u ^<£>£>£, rtti^n^-^^ i^^y 

[0 0 8 1 ] Kflt^fr^T* y fc^ coffin v-*^ 

M% U>y y >y-tZ>fz$>\Z. yam^—^CDyionte Carl 
o^h^-t^^^^T^6o ^&<Dmt LTIi, C 
ookftfe^ r#;fc5fc^ h U— i/^^(Distribution Ray Tra 
cing)j , Computer Graphics (SIGGRAPH '84 Proceedin 
gs), pp. 165-174, July 1984*5 «fc tfShirleyft&S rgg^ 
Rgl^ftg(7}fc#><7)Monte Carlo^fe (Monte Carlo techni 
ques for direct lighting calculations) J ^ Transact 
ions on Graphics, 15(1) : 1-36, 1996CHE® £ tlX 1/^-5 
hCOfrhZ* Z.<Djj&:\Z£Z> t. ffiE^OBRDF (bi-direc 
tional reflectance distribution functions : MJj&) 

R&mftttmm) Afflux, jnuft^*— 

I^St^-CfcSo «J3gW5t»»*i, 

SSSlt (diffuse interref lection) I^M^^tt^ § 

[0 0 8 2 ] mm<nWX&, ^.M^K^^y'^n. M 

m<nA?%<nm<Dm'i£\z&^^x\<^z>o fs^wtit 

tt, Marschnerftfe^ r AW i®«^<- ^ BRDF 

$'Ja5 (Image-based BRDF measurements including human 
skin) J x Rendering Techniques '99 (Proceedings of 
the Eurographics Workshop on Rendering), pp. 131— 
144, June 1999iCfS«c £ ftT ^ 5 h <Ofr &> 5 0 Ztlb<D 
9lfen. ^< od^tfi¥»<offl<OSp*&BRDF*iaiE LT*3 
D v Lafortuneftil^ rKSt^H^<0^i»Jgift(a^ (Non-1 i 
near approximation of reflectance functions) J x Co 
raputer Graphics (SIGGRAPH '97 Proceedings), pp. 117 
-126, August 1997|Cf2ic$ nr^^BRDF^e^/K^ii^ 

LT^^a^^^SJ^BRDF— Ojg^^^>^*!>, ^tti 
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^ * ^ ^ ^ y^St 13 Aix <b ixX t^o 
10 0 8 3] i£»Rl*/*5»<*>^* o£ 
9, rr/U^< Y-vy-f (albedo map) J aTtfflMt 

^^J7T^^f f v^/T' (scalartexture map) (C J: o 

6BRDFX&9, mtt, Lafortuneftfe^ fKI**IJ|»<& 
^$&7l£#r<EUb (Non-1 inear approximation of reflectan 
ce functions) J ^ Computer Graphics (SIGGRAPH '97 P 
roceedings), pp. 117-126, August IW7 (DWi&ik =j It 
4 *S P — >^ (generalized cosine lobes) C>?P 131 <£ o X 

[0 0 8 4] T /l"< K"^ > :/(7)$t^ 

[0 0 8 5] 13 1 1 12, ^^*^¥JC£fc«:lHfcS:Bi0 

(D^XfAT'll f^^/M^7l 100«^tl, 

&»<a3tea££#A,Xi>£ 0 -tcD^o< d^Wlil 10 
2, 110 4 X^£tiX^6 0 ir*l 1 0 

6, 1108, ftifflll 0<DMfo% LftSyt? 4 /l- 

££5I-LXV>5 0 r co^jTMi. -^>>8» 

^jtg^-y^ v&zf}) >- v^titz.m^i 1 1 2&mm 

^7111011 t6¥*dS[H»ELT^SPfl»il:U^:** 
g^ftfcTfcSI 1102, 1104 ^f)^TV^o 2t;ftfc 

[0 0 8 6] y "->a ytt^otP 

6<£>X. ffifei®, (irradianceh fi^*r6]. 

w fc#-C# 6„ JlSlftft 7 V^</1- h (Lambert ia 
n) SW^®^^n6^C0i®^(DTXML 7^^hS 
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2 0(0^1 (A) , (B) ft^Z *^V&mft\Z&B£ 

M/i* — ~?1fi : r $ x^ J r~**yy r fo(D-f'<X<DZ*#±/i'&m 
6 J: 5 Iwj® t) cF JX^iS?"^ :y t° l/tf (reverse mapping) 

(stochastic supersampl ing) Ifi&fflZtl 

10 X^5 P 

[0087] 7 m<n 

(grazing) ^ £ # f gll^o X i£o 

-r^xoasu^ ^^(D^Dfi^^j^ft-^^ttxv^o 

fi^f+ttBBJftl** t©^a7/m#Bl3 (A) , 
(B) K^^Xt^^, ®{-;#LX 15 ft It 

TP-6o (cos 0 i cos 8 e - c) p , ti fd L , c= 0. 2, p = 
[0 0 8 8] *&%<n s r 2 Xf-v £#±/U<DT/l"< K£tf 

MPt$^ S»«tifc5K ^±X&>Z>fr<D-* — * 

< ^ t) 1 iot^ciit^^ t'(Dytm\c 

[0 0 8 9] Ztbb<DmfeKM*&&htz1tZ>tzbblCte > 
^Tm&JEfc&mXfcZo # > 7<Dm^&mn. Debeve 

40 ~7<DliB\m (Recovering high dynamic range radiance ma 
ps from photographs) J , Computer Graphics (SIGGRAP 
H '97 Proceedings), pp. 369-378, August 1997^12® 
SttXv^-5*feS:fi6fflLX«lES^Xt^o 

K^)¥S^Si LXit£iE£ixXt><5 0 i/VX(D7t 

-^7/UU^^^^t>-r^fL ZhangS r^^^^IE^/c 
ft07U^'>^/<fi^j£(A flexible new technique 
for camera cal ibrat ion) J % Technical Report MSR-TR- 
98-71, Microsoft Research, l998(CfeH^ ^ tit I* 5 ^ 
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[0 0 9 0] Jitd.yv-t^fab'&bixti'rf J* 3 ?*-* y 

[00911014 (A), (B) i:H 1 5 (A) , 

(B) fix IB*ffi<&r/i"< K-^ y^i:ai807M K 

I 0 0 9 2 ] HI 1 6 —a 1 8 f+^-T5 tT^^^feO 
«*BS:^-ria-e*>-5o lfr@ (H16(D (A) , 

(b> , (o ) k« % ifr<Dftmfrt><Dm&7F$ 

ftt^S. TA"< s^fcSJSStt/EiBRDF 

2fr@ (HI 70 (A) . (B) , (C) ) «U ^U- 

u^y y v^lcmtt^C iri-RiK 3f?g (H 

is<d (a) , (b) , (o ) -m, mm<n&M&m 

[0 0 9 3] HI 9 12. rrtlft^Utt^IS^ll: 
€:fE«t LXi^Z>7 d-^^- h-C*>^ 0 if 

its t>o<t it, y 7 h ^x7x-is$nti^ 0 

[0 0 9 4] ^x^l 7 0 0T\ RSJf-T -5 
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7 0 2 T\ ®¥ft<D^te5fc^T-f8W£;ft,5o 
^7^>>^1 7 0 4 X\ £iir> *<orai--jg 

statBas, w.&jjfa&xxf'r* swift <&ffi« 

IB ft <o 4*36 ^t/H:^ttS7 v-</u hKW**5W-» £ 

fy^l7 12t% iBAntDfc^ir/u^T/u^ K^s^J- 
»o-Ct^^*rp A^ s «WrSix6. ^otl^ 
y^l 7 1 2X^<D \?9±/\'i£%hirh. ^fy/1 7 0 

fctttttf* xryyi 7Ut\ ii^JDO-r ^^Hitta* 
^ot^5^d:'5 fr&f&frftbtiZo ifi*Dtf>7 f v**/HB 
itf ^/J^oT^fttf, Xy"y?l 7 1 6 XVKDfis* ^® 

B«36s»o-ct^>k*ttttf. ^f^yi 7 1 8t% 

liS^WJ: 5{-ff*Ptt-5^<OAft:«fi. MarschnerS 

y^X^iJfrUV^y V^dnver 

se Rendering for Computer Graphics)] x PhD thesis, 
Cornell University, August 1998lwf£1jiJt ^ ixX 1/ N -6 0 
[0 0 9 5] H2 011, ->y^t^t/K^7;^Kt 

yi 8 0 0T% fc6t'^-tryu^^x.^ttTt>^ < b#. 
fitr^^n*^ (^7^1 8 0 4) o e4"tr/wds%£: 

^ U7«;/1 8 0 4) . t'^ir/u^^/^ < <i: t> 1 ocd 

^ 5 a*#*ij0rSix<& o t5-e«>iitf, ^ot^t/H:o 
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[ 0 0 9 6] fiL-h* Lfc«<D»««:r = *-->'3 > 
[ 0 0 9 7] 7^!)^r->3 p ^ 9 A fi C P U 

[0 0 9 8] S fefcT^y >5r— >3 y/D^7A^fT 
[0 0 9 9] 

fttmn. j:6ABBojft/9^BRDFass^e>»e>tt, 

hy, l^fcctt/SW^^tll^-^Ol/yy^ (phys 
ically-based Tenderer) "Cffiffl^^ ^ k J: o X 
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[0 10 0] EA±. i$i£±^®:fcJ;uV;2;fcte;fr&* 
[0®OfiS»^|ft^] 

[01] *asw<oiiiifijKiBS:iis-*--5±-e«effl-r5<o^ 

[02] S¥^^^oi?iitts$a^stt^m$a<!:^i^^(w 
si 9 a tf fc 1 w ^ J m x # ^ > * ^ ^ £ ^ -r mis. 0 x $> 

[0 3] *&m(ont&j&m^'ft<>xm&mnk&L&tmtii 

[04] **W<D3l»»lBlwJ:-6*!|*=»- K^s/^ 
[05] **Mo||»JKffi^J:sa«3C«|^o'tr^S:^ 

[0 6] *ftw<nnmMn<obz>m<Dmm&mm'*imx 

foZffl^i'^'r A^r^-T^P y^itfc^o 
[0 7] (A) - (C) tt*«W<0|ttliJB«*wiS«^ 

-tmxhZo 

[0 8] s/v^fc, w^/j;^:^ 

-So 

[0 9] (A) *5cfct/ (B) l*ffiig^i@gTM#»^*tJ& 

^ tz&\zmm£ftz>m^Mm%:7f;-fmxfoz> 0 

[0 1 0] *^P^oHffi^ffi(cJ:-5®ig^^a<D^x$/ 
:/Sr8B*fc Ltt^7 p-^^- ht'fo^o 
[011] ^WO^MiCftota^r/M y 

y&tiFZfcfrizmmxz &w^^*T^z^-tmj$.mx 

[01 2] (A) *3 J:t5 (B) ttT^^YSftlrtlCtft 

[0 13] (A) *3j:a (B) IJH1 2«w^-T¥*^» 
[014] (A) (B) U2o(OS«57-^t 

[01 5] (A) fectt/ (B) dSilfcfct(O0 1 4 

[016] (a) - (o temtezmw&ftnTxgite 

izis^y y >y $titiMffsi'<D=* is? >3 
m-rmx&>z> 0 

[017] (a) - (o nmtezmw&ftoTxmti: 
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